AI Platform
The AI platform turns accelerated infrastructure into reusable services for model and application teams. The core defines capabilities and interfaces. Implementations select supported products and versions.
Explore AI platform ecosystem solutions.
Build and Customize
The lifecycle must support:
- Approved model, dataset, prompt, tool, and evaluation inputs.
- Reproducible training or customization jobs.
- Immutable artifacts with provenance and promotion state.
- Separation of development, testing, and production authority.
- Rollback or withdrawal when a release no longer meets requirements.
NVIDIA implementation options include NVIDIA NeMo for model workflows. The broader NVIDIA Agent Toolkit includes NeMo Fabric for agent integration and execution, and NeMo Relay for runtime telemetry and policy integration. Select supported components that satisfy the architectural contract.
Evaluate and Promote Continuously
Evaluation is not a one-time development gate. Apply quality, safety, security, policy, and operational evaluation throughout the lifecycle:
- Evaluate candidates against representative task, safety, security, and policy scenarios before promotion.
- Preserve the evaluation set, configuration, result, owner, and asset versions as linked evidence.
- Correlate production outcomes, agent traces, drift, incidents, latency, and cost with the deployed asset version.
- Re-evaluate when models, prompts, tools, data, policy, runtimes, or dependent services change.
- Hold, roll back, or withdraw releases that no longer meet approved thresholds.
NeMo Evaluator is an implementation option for evaluation. NeMo Relay supplies agent telemetry for evaluation and observability systems. AgentOps and Observability supply production evidence. Named owners retain promotion and risk decisions.
Optimize and Serve
Inference services expose consistent model discovery, admission, request, streaming, error, health, and telemetry contracts. Runtime selection follows workload objectives and the selected hardware foundation.
Inference Applications define application demand and service objectives. Optimize and Serve satisfies that demand through model optimization, runtime selection, deployment, scaling, and serving. It does not own application behavior, business outcomes, or enterprise API policy.
Implementation options can include NVIDIA NIM, NVIDIA Dynamo, Triton Inference Server, and TensorRT-LLM. Product inclusion here does not make any one runtime mandatory.
Protect Model IP and Inference Confidentiality
Protect model intellectual property and the confidentiality of inference inputs, retrieved context, and outputs across loading, optimization, serving, caching, and telemetry. Restrict model-artifact access and avoid exposing sensitive request content through logs or traces.
Governance and Assurance defines the protection objectives and acceptance evidence. Platform Services enforces identity, access, and disclosure policy. Select confidential-computing mechanisms only when the threat model and supported deployment require them.
NVIDIA Software Decision Guide
Use this table to distinguish functional roles. It is not a compatibility statement or required bundle. Confirm the target release, GPU, operating system, hypervisor or cloud, Kubernetes distribution, and dependent component versions. Use the NVIDIA AI Enterprise Infrastructure Support Matrix and each product’s current documentation.
Orchestrate Resources
Resource orchestration must define:
- Queues, quotas, priorities, preemption, and fairness.
- Placement constraints and topology awareness.
- Whole-GPU, partitioned-GPU, and time-sharing policies where supported.
- Tenant and project capacity boundaries.
- Scheduling signals, utilization objectives, and chargeback inputs.
- Behavior during maintenance, capacity loss, and recovery.
NVIDIA Run:ai is one implementation option for dynamic GPU orchestration.
Scheduling and Accelerator Ownership
Use the following ownership boundaries to separate business policy, Kubernetes scheduling, accelerator enablement, and physical infrastructure:
In a Kubernetes-native path, kube-scheduler filters, scores, and binds Pods. With an integrated scheduler such as NVIDIA Run:ai, policy and scheduling responsibilities can extend into that path. Record the selected scheduler, its authority, and its interaction with Kubernetes instead of assuming one universal owner.
Refer to the Kubernetes Scheduling Framework and kube-scheduler descriptions for the default pod-scheduling boundary. Refer to the NVIDIA Run:ai and NVIDIA GPU Operator documentation for implementation-specific orchestration and accelerator-software lifecycle scope.
Interfaces to Adjacent Layers
The AI Platform exchanges the following contracts with adjacent layers: