> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/ai-enterprise/ai-factory/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/ai-enterprise/ai-factory/_mcp/server.

# AI Platform

> Implementation-flexible AI platform capabilities for model and agent development, continuous evaluation, optimization, serving, and resource orchestration.

The AI platform turns accelerated infrastructure into reusable services for model and application teams. The core defines capabilities and interfaces. Implementations select supported products and versions.

Explore [AI platform ecosystem solutions](/ai-enterprise/ai-factory/ecosystem-integrations/solution-catalog?layer=ai-platform).

## Build and Customize

The lifecycle must support:

* Approved model, dataset, prompt, tool, and evaluation inputs.
* Reproducible training or customization jobs.
* Immutable artifacts with provenance and promotion state.
* Separation of development, testing, and production authority.
* Rollback or withdrawal when a release no longer meets requirements.

NVIDIA implementation options include [NVIDIA NeMo](https://docs.nvidia.com/nemo/) for model workflows. The broader NVIDIA Agent Toolkit includes [NeMo Fabric](https://docs.nvidia.com/nemo/fabric/about-nemo-fabric/overview) for agent integration and execution, and [NeMo Relay](https://docs.nvidia.com/nemo/relay/about-nemo-relay/overview) for runtime telemetry and policy integration. Select supported components that satisfy the architectural contract.

## Evaluate and Promote Continuously

Evaluation is not a one-time development gate. Apply quality, safety, security, policy, and operational evaluation throughout the lifecycle:

* Evaluate candidates against representative task, safety, security, and policy scenarios before promotion.
* Preserve the evaluation set, configuration, result, owner, and asset versions as linked evidence.
* Correlate production outcomes, agent traces, drift, incidents, latency, and cost with the deployed asset version.
* Re-evaluate when models, prompts, tools, data, policy, runtimes, or dependent services change.
* Hold, roll back, or withdraw releases that no longer meet approved thresholds.

[NeMo Evaluator](https://docs.nvidia.com/nemo/evaluator/latest/) is an implementation option for evaluation. [NeMo Relay](https://docs.nvidia.com/nemo/relay/about-nemo-relay/overview) supplies agent telemetry for evaluation and observability systems. AgentOps and Observability supply production evidence. Named owners retain promotion and risk decisions.

## Optimize and Serve

Inference services expose consistent model discovery, admission, request, streaming, error, health, and telemetry contracts. Runtime selection follows workload objectives and the selected hardware foundation.

[Inference Applications](/ai-enterprise/ai-factory/core-reference-design/workloads#representative-workload-families) define application demand and service objectives. Optimize and Serve satisfies that demand through model optimization, runtime selection, deployment, scaling, and serving. It does not own application behavior, business outcomes, or enterprise API policy.

Implementation options can include [NVIDIA NIM](https://docs.nvidia.com/nim/), [NVIDIA Dynamo](https://docs.nvidia.com/dynamo/latest/), [Triton Inference Server](https://docs.nvidia.com/deeplearning/triton-inference-server/), and [TensorRT-LLM](https://docs.nvidia.com/tensorrt-llm/index.html). Product inclusion here does not make any one runtime mandatory.

### Protect Model IP and Inference Confidentiality

Protect model intellectual property and the confidentiality of inference inputs, retrieved context, and outputs across loading, optimization, serving, caching, and telemetry. Restrict model-artifact access and avoid exposing sensitive request content through logs or traces.

[Governance and Assurance](/ai-enterprise/ai-factory/core-reference-design/security-and-governance#model-and-inference-confidentiality) defines the protection objectives and acceptance evidence. [Platform Services](/ai-enterprise/ai-factory/core-reference-design/platform-services#security-responsibility-boundaries) enforces identity, access, and disclosure policy. Select confidential-computing mechanisms only when the threat model and supported deployment require them.

## NVIDIA Software Decision Guide

Use this table to distinguish functional roles. It is not a compatibility statement or required bundle. Confirm the target release, GPU, operating system, hypervisor or cloud, Kubernetes distribution, and dependent component versions. Use the [NVIDIA AI Enterprise Infrastructure Support Matrix](https://docs.nvidia.com/ai-enterprise/support-matrix/latest/index.html) and each product's current documentation.

| Component                                                                                | Primary Role                                                          | Use When                                                                             | Key Dependencies and Boundary                                                                                                        |
| ---------------------------------------------------------------------------------------- | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------ |
| [NVIDIA NeMo](https://docs.nvidia.com/nemo/)                                             | Model development, customization, evaluation, and guardrail workflows | Teams need NVIDIA-supported model and data lifecycle capabilities                    | Requires governed data, compute, artifacts, evaluation policy, and promotion. It is not the serving control plane.                   |
| [NVIDIA NIM](https://docs.nvidia.com/nim/)                                               | Packaged model inference endpoints                                    | A supported model needs a consistent containerized serving interface                 | Requires supported GPU and runtime infrastructure. Service routing, tenancy, and enterprise policy remain adjacent responsibilities. |
| [NVIDIA Dynamo](https://docs.nvidia.com/dynamo/latest/)                                  | Distributed inference execution and scaling                           | Serving needs disaggregated or distributed execution across accelerated capacity     | Requires compatible model runtimes, networking, scheduling, and telemetry. It does not replace enterprise API policy.                |
| [Triton Inference Server](https://docs.nvidia.com/deeplearning/triton-inference-server/) | Multi-framework model serving protocols and scheduling                | Applications need a general inference server across supported backends               | Requires compatible model repositories and runtimes. Fleet routing and tenant governance remain outside the server.                  |
| [TensorRT-LLM](https://docs.nvidia.com/tensorrt-llm/index.html)                          | Large language model (LLM) runtime optimization                       | LLM latency, throughput, memory, or scale objectives require NVIDIA GPU optimization | Requires model- and GPU-specific build and runtime compatibility. It is not an API gateway or lifecycle system.                      |
| [NVIDIA Run:ai](https://docs.nvidia.com/run-ai/index.html)                               | GPU scheduling, quotas, and workload orchestration                    | Shared accelerated clusters need governed allocation and utilization controls        | Requires a supported platform and resource model. Application service objectives and model lifecycle remain separate.                |

## Orchestrate Resources

Resource orchestration must define:

* Queues, quotas, priorities, preemption, and fairness.
* Placement constraints and topology awareness.
* Whole-GPU, partitioned-GPU, and time-sharing policies where supported.
* Tenant and project capacity boundaries.
* Scheduling signals, utilization objectives, and chargeback inputs.
* Behavior during maintenance, capacity loss, and recovery.

[NVIDIA Run:ai](https://docs.nvidia.com/run-ai/index.html) is one implementation option for dynamic GPU orchestration.

## Scheduling and Accelerator Ownership

Use the following ownership boundaries to separate business policy, Kubernetes scheduling, accelerator enablement, and physical infrastructure:

| Responsibility                                                                                             | Owner in This Design                                                                                    | Boundary                                                                                                                                                                                                                        |
| ---------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Business priority, project quota, fairness, queue policy, and topology intent                              | [Resource Orchestration](/ai-enterprise/ai-factory/core-reference-design/ai-platform#orchestrate-resources)                      | Translates workload and organizational policy into placement constraints. The selected implementation can also extend or participate in Pod scheduling and binding.                                                             |
| Default Pod filtering, scoring, binding, reconciliation, and node assignment                               | [Enterprise Kubernetes](/ai-enterprise/ai-factory/core-reference-design/cloud-native-platform#ownership-boundary)                | Supplies the default scheduling path against declared constraints and allocatable resources. An integrated scheduler can extend or participate in this path. Kubernetes does not define business priority or physical topology. |
| GPU drivers, device plugins, container toolkit, node labels, telemetry, and accelerator software lifecycle | [Accelerator Operators](/ai-enterprise/ai-factory/core-reference-design/cloud-native-platform#nvidia-operators)                  | Reconciles accelerator enablement and exposes resources to Kubernetes. It does not choose workload priority or quota.                                                                                                           |
| Physical systems, fabrics, storage, supported scale points, and topology                                   | [Enterprise Hardware Reference Architectures](/ai-enterprise/ai-factory/core-reference-design/enterprise-hardware-ra-foundation) | Defines infrastructure capabilities and supported designs. It does not perform software scheduling.                                                                                                                             |

In a Kubernetes-native path, kube-scheduler filters, scores, and binds Pods. With an integrated scheduler such as NVIDIA Run:ai, policy and scheduling responsibilities can extend into that path. Record the selected scheduler, its authority, and its interaction with Kubernetes instead of assuming one universal owner.

Refer to the Kubernetes [Scheduling Framework](https://kubernetes.io/docs/concepts/scheduling-eviction/scheduling-framework/) and [kube-scheduler](https://kubernetes.io/docs/concepts/scheduling-eviction/kube-scheduler/) descriptions for the default pod-scheduling boundary. Refer to the [NVIDIA Run:ai](https://docs.nvidia.com/run-ai/index.html) and [NVIDIA GPU Operator](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/) documentation for implementation-specific orchestration and accelerator-software lifecycle scope.

## Interfaces to Adjacent Layers

The AI Platform exchanges the following contracts with adjacent layers:

| Interface                                                                       | Contract                                                                      |
| ------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- |
| [Workloads](/ai-enterprise/ai-factory/core-reference-design/workloads)                                   | Demand, quality, latency, throughput, tenancy, and availability objectives.   |
| [Platform services](/ai-enterprise/ai-factory/core-reference-design/platform-services)                   | Identity, gateway, connectors, policy, artifacts, delivery, and telemetry.    |
| [Cloud-Native Platform](/ai-enterprise/ai-factory/core-reference-design/cloud-native-platform)           | Scheduling primitives, runtimes, networking, storage, secrets, and operators. |
| [Hardware foundation](/ai-enterprise/ai-factory/core-reference-design/enterprise-hardware-ra-foundation) | Compute, memory, network, storage, scale, and availability envelope.          |