Architecture and Components

NVIDIA Run:ai is a Kubernetes-native platform for deploying, scheduling, and managing AI workloads across shared GPU infrastructure. It extends Kubernetes with AI-specific capabilities for resource management, scheduling, governance, and workload lifecycle management, while Kubernetes remains the underlying orchestration layer. Run:ai runs on standard Kubernetes clusters backed by NVIDIA GPU infrastructure: GPU nodes, high-speed interconnects (NVLink, InfiniBand, RoCE), and high-performance shared storage.

The platform is organized into three layers that work together: the interfaces used to interact with the platform, a control plane that centrally manages configuration and state, and cluster components that enforce that state on each Kubernetes cluster running workloads.

User Interfaces

NVIDIA Run:ai exposes four interfaces to administrators, AI practitioners, and external systems: the web UI, CLI, REST API, and MCP server. All four provide access to the same underlying platform capabilities: submitting and managing workloads, administering the platform, and integrating with external tools and automation.

• Web UI - The primary interface for administrators and AI practitioners. Submit and monitor workloads, manage organizational structure and access, and view usage and performance data.
• CLI - Submits and manages workloads and queries cluster state from the command line, for interactive use or scripting.
• REST API - The programmatic interface underlying the UI and CLI, used to integrate NVIDIA Run:ai with external tools and automated pipelines.
• MCP Server - Exposes NVIDIA Run:ai control-plane operations as tools for AI agents and assistants (Claude Code, Cursor, etc.) via the Model Context Protocol, paired with Agent Skills for guided, task-specific workflows.

Control Plane Services

The control plane is the centralized management layer of NVIDIA Run:ai. It provides the APIs and services that configure, monitor, and manage AI environments across one or more Kubernetes clusters from a single place, and because it is decoupled from any single cluster, one control plane can manage multiple clusters and tenants, with each tenant getting its own isolated UI, API, and CLI. Internally, control plane services are independently deployable microservices that communicate over a pub/sub event bus. The sections below group these services by the capability they deliver - identity and access, organizational structure, workload lifecycle, observability, multi-cluster management, and platform delivery. See the Service Reference for the complete list of control-plane services.

• Identity and Access - Handles authentication, identity provider integration, and RBAC enforcement across all services. Every request to NVIDIA Run:ai, through the UI, CLI, or API, is authenticated and then authorized before it is allowed to proceed. gateway-hub is the entry point for every request, validating the bearer token and resolving the tenant from the request subdomain before routing it onward. identity-manager then authenticates the requester, resolving external identity providers, local users, and service accounts, and issuing and revoking tokens. authorization evaluates that identity against RBAC roles and access rules to resolve the effective permissions for the request.
• Organizational Structure - Manages the hierarchy of tenants, departments, and projects that scopes resource quotas and access rules. Compute is organized into a physical layer (nodes and node pools) and an organizational layer (departments and projects) connected through quota. tenants-manager manages tenant lifecycle, creation, deletion, and configuration, in multi-tenant deployments, giving each tenant its own isolated UI, API, and CLI. org-unit-service persists the department and project hierarchy and the quota allocated at each level, and syncs it to every managed cluster so scheduling and quota enforcement always resolve against the same source of truth.
• Workload Lifecycle - Handles workload submission, policy enforcement, asset management, and persistent storage. policy-service stores workload policies per scope, tenant, department, project, and computes the effective policy for each. When a workload is submitted, workloads-manager validates it against that effective policy, resolves any assets it references, and generates the Kubernetes manifests that get synced down to the cluster. assets-service manages those reusable assets, environments, compute resources, credentials, data sources, so workloads can reference a standardized configuration instead of repeating it.
• Observability - Collects and serves GPU and workload metrics, audit logs, and event-driven notifications. metrics-service queries per-tenant, per-cluster Prometheus stores and serves GPU and workload metrics over REST. audit-service ingests user and system action events from the pub/sub bus and persists them for query. notifications-service ingests platform events and routes and renders them via configurable rules and templates, dispatching to channels such as email and Slack.
• Multi-Cluster Management - Allows one control plane to manage workloads and resources across many connected Kubernetes clusters. cluster-service maintains control-plane state for cluster-scoped resources, currently node inventory, and syncs it from every connected cluster so cross-cluster views stay current.
• Platform Delivery - Serves the web UI and CLI binaries. frontend is the web UI.

Cluster Components

Each Kubernetes cluster connected to NVIDIA Run:ai runs a set of cluster components that extend Kubernetes with AI-specific capabilities, installed and managed by the go-operator, which owns their full lifecycle: deploy, configure, and update. The cluster plane is CRD-driven: components reconcile control-plane desired state into cluster CRDs such as RunaiConfig, AccessRule, and WorkloadRunner, and report status back through the same CRDs. See the Service Reference and CRD Reference for the complete list of cluster components and CRDs.

• Data Sync - agent pulls desired state for every resource kind, projects, workloads, access rules, node pools, credentials, from the control plane and reconciles each into the cluster. cluster-sync watches those CRDs and live Kubernetes objects and pushes their state back up to the control plane. This two-channel pattern, desired state down, live state up, is how the two planes stay synchronized, over an outbound-only, secured connection so clusters never need to be publicly reachable.
• Scheduling - admission-controller intercepts pod creation and injects Run:ai scheduling attributes, labels, annotations, resource patches, before a pod is admitted. The actual placement decision is made by the Scheduler, built on the external KAI Scheduler (see External Dependencies below), which applies fair-share, preemption, bin-packing, and topology-aware placement rules. Each node pool runs its own Scheduler instance, so scheduling decisions for one hardware group never compete with another. Once a workload is running, runtime-rules enforces time-based policies, terminating interactive and training workloads that exceed their configured idle or time limits.
• Workload Lifecycle - anyworkload-controller translates a workload’s CRD into its underlying pods and manages their full lifecycle: creation, status, teardown.
• RBAC and Project Isolation - accessrule-controller reconciles AccessRule CRDs into native Kubernetes RoleBindings in the corresponding project namespace. shared-objects-controller replicates shared objects, primarily PVCs from DataVolume CRDs, into every project or department namespace that references them.

Architectural Flow

A request moves through the platform in a loop: down from the control plane to the cluster, through execution, and back up as status.

1. A user or external system submits a request through the web UI, CLI, REST API, or MCP server.
2. The control plane authenticates the request and updates the desired state it owns: the workload itself, plus any organizational resources, policies, or configuration it depends on.
3. Cluster components sync that desired state down to the relevant Kubernetes cluster and reconcile it into Kubernetes resources.
4. The admission controller and Scheduler validate the request, apply policy, and decide where and how the workload runs.
5. Kubernetes runs the workload on the available compute, GPU, networking, and storage infrastructure.
6. Cluster components watch workload and cluster state and sync status back up to the control plane.
7. The control plane surfaces that updated state through the NVIDIA Run:ai interfaces, closing the loop with centralized visibility across every connected cluster.

External Dependencies

NVIDIA Run:ai’s cluster components build on external open-source projects, pulled in as versioned Helm chart dependencies or Go modules rather than built from Run:ai source. NVIDIA Run:ai packages, configures, and manages these dependencies as part of the platform, so no separate installation or maintenance is required to use them.

• KAI Resource Management (KRM) - An open-source Kubernetes-native resource-management layer for KAI Scheduler, adding projects, departments, node pools, queues, and workload-placement abstractions. Its Helm chart packages nodepool-controller, project-controller, pod-group-assigner, and krm-operator, replacing the versions of those controllers previously built from Run:ai’s own cluster source. KRM is a direct Helm chart dependency of runai-chart.
• KAI Scheduler - An open-source, CNCF-donated AI-aware scheduling engine that applies fair-share, preemption, bin-packing, priorities, gang and topology-aware scheduling, and quota and fairness rules to GPU workloads. It is the engine underlying the NVIDIA Run:ai Scheduler, integrated with Run:ai-specific configuration, and is a sub-chart dependency of KRM.
• gpu-fractioning - A Kubernetes operator that enables multiple pods to safely share a single GPU with enforced memory boundaries. It runs alongside KAI Scheduler: KAI Scheduler decides which fraction of which GPU a workload gets, and gpu-fractioning enforces that memory boundary on the node and exports per-pod metrics for the resulting fractional GPUs. It is a sub-chart dependency of KAI Scheduler.
• kai-resource-management-api - The API-only counterpart to KRM: the CRD type definitions (kai.resources: Project, Department, NodePool, ManagedNodesConfig, KRMConfig). Consumed as a Go module across Run:ai’s own cluster services that read or write these CRDs, independent of which controller owns reconciling them. Migrating legacy CRDs onto kai.resources runs automatically this release and self-gates once complete; it is not a customer-facing toggle.
• workload-map - A CRD-based resource abstraction layer with its own operator, and a direct Helm chart dependency of runai-chart. Karta defines a standard schema for describing any Kubernetes workload type - PyTorchJob, RayCluster, JobSet, and custom CRDs - so controllers can extract pod specs, replica counts, and status without per-type logic. Used by NVIDIA Run:ai’s anyworkload-controller to create the pods for a submitted workload.