Configuration

This page is for Self-Hosted, version v3.0 (default). For other versions, use one of these documentation indexes: - v3.0 (default): https://docs.nvidia.com/run-ai/self-hosted/v3.0/llms.txt - v2.26: https://docs.nvidia.com/run-ai/self-hosted/v2.26/llms.txt - v2.25: https://docs.nvidia.com/run-ai/self-hosted/v2.25/llms.txt - v2.24: https://docs.nvidia.com/run-ai/self-hosted/v2.24/llms.txt

Day 1 Configuration covers the infrastructure configuration that makes NVIDIA Run:ai production-ready for your environment, once installation is complete. Which configurations apply depends on your environment and workload needs - not every section here is required for every deployment.

Cluster and Control Plane Settings

• Node Roles - Designates dedicated nodes for NVIDIA Run:ai system services, GPU workloads, and CPU-only workloads, so system services do not compete with GPU workloads for CPU cycles on expensive GPU machines.
• Advanced Control Plane Configuration - Advanced Helm chart values for customizing the control plane deployment beyond its defaults.
• Advanced Cluster Configuration - Advanced Helm chart and runaiconfig values for customizing cluster behavior - scheduling policies, resource management, and system service tuning.
• Cluster Authentication - Configures the Kubernetes API server to authenticate users via the NVIDIA Run:ai identity provider (OIDC), so users can securely submit workloads directly with kubectl.

Networking

• Kubernetes Gateway API - An alternative to Ingress for routing external traffic, offering a more flexible and extensible routing model.
• Service Mesh - Configuration required to run NVIDIA Run:ai alongside a service mesh with sidecar injection.
• External Access to Containers - Lets researchers connect to running containers remotely - for example, to run a Jupyter Notebook or connect an IDE for remote development.

Security

• User Identity in Containers - Propagates the correct user identity into containers instead of running as root by default, which matters for both functionality (like network file system access) and security.
• Security Best Practices - Actionable, platform-native and Kubernetes-mapped best practices for securely configuring and operating NVIDIA Run:ai.
• Secure Your Cluster - The specific permissions NVIDIA Run:ai requires, for administrators and security officers evaluating the platform.

Storage

• Storage - Sets up shared storage accessible to every worker node, so distributed training and shared datasets can be read and written consistently across nodes.

Third-Party Integrations

• Supported Integrations - Kubernetes components and external tools with built-in NVIDIA Run:ai support.
• Community Support Integrations - Integrations commonly used alongside NVIDIA Run:ai but not supported out of the box.
• Karpenter - Guidance for running the Kubernetes cluster autoscaler Karpenter (optimizes for cost) alongside the NVIDIA Run:ai Scheduler (optimizes for fairness and utilization), since the two can conflict.