Installation

Installation Guide

Install accelerator support and the Dynamo Platform on Kubernetes.
View as Markdown

This guide walks you through installing everything needed to deploy models with Dynamo on Kubernetes. Follow the steps in order — each builds on the previous one.

Prerequisites

Before you begin, make sure you have Helm v3 or later and choose the accelerator type in your cluster:

The GPU Operator in the first installation step can install NVIDIA drivers. Do not also enable provider-managed drivers when you create the GPU node pool. If the nodes already have drivers, disable GPU Operator driver management as shown in that step.

Verify the client tools:

$kubectl version --client
$helm version

Overview

Every Dynamo deployment requires accelerator support and the Dynamo Platform. NVIDIA GPU clusters can use the NVIDIA GPU Operator. Intel GPU clusters use the Intel resource driver with DRA. The Dynamo Platform installation is the same after the cluster exposes its accelerator resources. Everything else is optional.

Optional ComponentWhen you need itRequired for
Grove + KAI SchedulerMultinode or disaggregated inferenceMultinode deployments (operator errors without Grove or LWS)
Network Operator / RDMADisaggregated inference in productionAcceptable KV cache transfer performance (TCP fallback has ~200-500x degradation)
kube-prometheus-stackAutoscaling, metrics dashboards, or the PlannerPlanner sla mode, KEDA/HPA autoscaling
Shared storage (model cache)Large models (>70B) or many replicasAvoiding per-pod downloads and HuggingFace rate limits

Grove + KAI Scheduler — Grove is the default multinode orchestrator. The operator returns a hard error on multinode deployments if neither Grove nor LeaderWorkerSet (LWS) is available. KAI Scheduler is optional but recommended alongside Grove for GPU-aware scheduling. See Grove for details.

Network Operator / RDMA — Without RDMA, disaggregated inference falls back to TCP automatically, but with severe performance degradation (~98s TTFT vs ~200-500ms with RDMA). Required for any production disaggregated deployment. Setup is cloud-provider-specific — see the Disaggregated Communication Guide and your cloud provider guide.

kube-prometheus-stack — Required for the Planner’s sla optimization mode (it reads live TTFT/ITL metrics from Prometheus). Also required for KEDA/HPA-based autoscaling. The Planner’s throughput mode can function without it using internal queue depth signals, but metrics-driven features will not work. See Metrics for details.

Shared storage — Prevents each pod from downloading model weights independently. Without it, large models (>70B) take hours to download per pod, and many replicas will hit HuggingFace rate limits. Not enforced by the operator — this is an operational concern. See Model Caching for the full walkthrough.

1

Install accelerator support

The NVIDIA GPU Operator automates deployment of the drivers, container toolkit, device plugin, and monitoring components used by NVIDIA GPU nodes.

$helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
$helm repo update
$
$helm install gpu-operator nvidia/gpu-operator \
>  --namespace gpu-operator \
>  --create-namespace
$  # Add a trailing \ above before uncommenting the following setting.
$  # --set driver.enabled=false

Set driver.enabled=false when the nodes already have provider-managed NVIDIA drivers. See the AKS, EKS, or GKE guide for provider-specific requirements.

Verify the installation:

$kubectl get pods --namespace gpu-operator
2

Install the Dynamo Platform

Set your environment variables:

$export NAMESPACE=dynamo-system
$export RELEASE_VERSION=1.2.1  # match a version from https://github.com/ai-dynamo/dynamo/releases
$helm fetch https://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/dynamo-platform-$RELEASE_VERSION.tgz
$helm install dynamo-platform dynamo-platform-$RELEASE_VERSION.tgz \
>  --namespace $NAMESPACE \
>  --create-namespace
$  # Note: add \ to --create-namespace above when uncommenting any optional flags below
$  #
$  # Grove + KAI Scheduler — uncomment if using multinode or disaggregated inference.
$  # Option A (install=true): Dynamo installs and manages Grove/KAI as bundled subcharts (dev/testing):
$  # --set "global.grove.install=true" \
>  # --set "global.kai-scheduler.install=true" \
>  # Option B (enabled=true): Grove/KAI are already installed externally (production):
$  # --set "global.grove.enabled=true" \
>  # --set "global.kai-scheduler.enabled=true" \
>  #
$  # kube-prometheus-stack — uncomment if Prometheus is installed (required for Planner sla mode and autoscaling):
$  # --set "dynamo-operator.dynamo.metrics.prometheusEndpoint=http://prometheus-kube-prometheus-prometheus.monitoring.svc.cluster.local:9090"

All helm install commands can be customized with your own values file: helm install ... -f your-values.yaml

Shared/Multi-Tenant Clusters: If a cluster-wide Dynamo operator is already running, do not install another one. Check with:

$kubectl get clusterrolebinding -o json | \
>  jq -r '.items[] | select(.metadata.name | contains("dynamo-operator-manager")) |
>  "Cluster-wide operator found in namespace: \(.subjects[0].namespace)"'

Namespace-restricted mode (namespaceRestriction.enabled=true) is only for development and testing. It is not supported for production. Set dynamo-operator.upgradeCRD=false; see Dynamo Operator.

Verify the Dynamo platform is running:

$# Check CRDs
$kubectl get crd | grep dynamo
$# Expected: dynamographdeployments, dynamocomponentdeployments, dynamographdeploymentrequests, etc.
$
$# Check operator and platform pods
$kubectl get pods -n $NAMESPACE
$# Expected: dynamo-operator-* and nats-* pods all Running
3

Install Optional Components

The Dynamo install command above includes commented flags for each optional component. Install the component first, then uncomment the corresponding flag before running helm install in Step 2 (or run helm upgrade --reuse-values with the flag if you’ve already installed Dynamo).

Multinode:

Multinode deployments require either Grove + KAI Scheduler or an alternative orchestrator setup (LeaderWorkerSet + Volcano) to enable gang scheduling for workloads that span multiple nodes. See the Multinode Deployment Guide for details on orchestrator selection and configuration.

Grove + KAI Scheduler

There are two ways to enable Grove and KAI Scheduler, controlled by which flags you uncomment in the Dynamo install command:

  • install=true — Dynamo installs and manages Grove/KAI as bundled subcharts. Simplest path; recommended for dev/testing.
  • enabled=true — Tells Dynamo that Grove/KAI are already installed and externally managed. Use this when you install Grove/KAI separately (e.g., to manage their lifecycle independently or share them across namespaces). Recommended for production.

For the enabled=true path, install Grove and KAI Scheduler separately first. See the Grove installation guide and KAI Scheduler deployment guide for instructions.

Compatibility matrix:

dynamo-platformkai-schedulerGrove
1.0.x>= v0.13.0>= v0.1.0-alpha.6
1.1.x>= v0.13.4>= v0.1.0-alpha.8
1.3.x>= v0.13.4>= v0.1.0-alpha.8, < v0.1.0-alpha.9
1.4.x>= v0.13.4>= v0.1.0-alpha.10

Upgrade Grove in lockstep with Dynamo while the Grove APIs remain unstable. Dynamo 1.3.x expects Grove’s earlier ClusterTopology API and is incompatible with the newer ClusterTopologyBinding API. Dynamo 1.4.x expects ClusterTopologyBinding.

LWS + Volcano

If you are not using Grove for multinode, you can use LeaderWorkerSet (LWS) (>= v0.7.0) with Volcano for gang scheduling. Both must be installed before deploying multinode workloads.

  1. Install Volcano:
$helm repo add volcano-sh https://volcano-sh.github.io/helm-charts
$helm repo update
$helm install volcano volcano-sh/volcano -n volcano-system --create-namespace
  1. Install LWS (>= v0.7.0) with Volcano gang scheduling enabled:
$export LWS_VERSION=0.8.0
$helm install lws oci://registry.k8s.io/lws/charts/lws \
>  --version=$LWS_VERSION \
>  --namespace lws-system \
>  --create-namespace \
>  --set gangSchedulingManagement.schedulerProvider=volcano \
>  --wait --timeout 300s

See the LWS docs and Volcano docs for configuration options, and the Multinode Deployment Guide for orchestrator selection.

Network Operator / RDMA

RDMA setup is cloud-provider-specific. See the Disaggregated Communication Guide for transport options, UCX configuration, and performance expectations, and your cloud provider guide for setup instructions:

kube-prometheus-stack

Install Prometheus before running the Dynamo install command so you can set the endpoint in one pass:

$helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
$helm repo update
$
$helm install prometheus prometheus-community/kube-prometheus-stack \
>  --namespace monitoring --create-namespace \
>  --set prometheus.prometheusSpec.podMonitorSelectorNilUsesHelmValues=false \
>  --set-json 'prometheus.prometheusSpec.podMonitorNamespaceSelector={}' \
>  --set-json 'prometheus.prometheusSpec.probeNamespaceSelector={}'

Then uncomment the prometheusEndpoint line in the Dynamo install command. The Dynamo operator automatically creates PodMonitors for its components. See Metrics for dashboard setup and available metrics, and Logging for the Grafana Loki + Alloy logging stack.

Shared Storage for Model Caching

Set up a ReadWriteMany PVC so all pods share downloaded model weights instead of each downloading independently. No Dynamo chart flags are needed — storage is configured in your deployment spec. Setup is cloud-provider-specific:

For large clusters with frequent model updates, consider ModelExpress for P2P model distribution and ModelStreamer for direct streaming from object storage. See Model Caching for the full walkthrough including the download Job, mount configuration, and ModelExpress setup.

4

Pre-Deployment Check

Run the pre-deployment check script to validate your cluster is ready for deployments:

$./deploy/pre-deployment/pre-deployment-check.sh

This checks kubectl connectivity, default StorageClass configuration, GPU node availability, and GPU Operator status. See Pre-Deployment Checks for details.

Alternative: Install with AICR

Steps 1–3 install each component — GPU Operator, Dynamo Platform, Grove, RDMA — with its own Helm command. NVIDIA AI Cluster Runtime (AICR) is an alternative that generates one version-locked recipe for the whole stack, then renders it into deployment-ready bundles for Helm, Argo CD, Flux, or Helmfile. For a supported environment, the aicr CLI drives the entire installation flow from a single recipe: GPU Operator, NVIDIA DRA driver, Network Operator (RDMA), Grove, KAI Scheduler, cert-manager, kube-prometheus-stack, and the Dynamo Platform.

AICR is a new, separately maintained project. It validates specific combinations of cloud, GPU, and OS, so it fits a cluster that matches a validated Dynamo recipe rather than an arbitrary environment. You still bring your own Kubernetes cluster — AICR generates and validates the runtime configuration, it does not provision clusters. See the AICR documentation for the authoritative support matrix.

AICR validates recipes across these dimensions; not every combination has a Dynamo recipe, so check the AICR docs for the validated set:

DimensionSupported values
Cloud / serviceAKS, BCM, EKS, GKE, Kind, LKE, OKE
GPUA100, B200, GB200, H100, H200, L40, RTX PRO 6000
Operating systemAmazon Linux, COS, RHEL, Talos, Ubuntu

Dynamo recipes use Dynamic Resource Allocation (DRA) for GPUs, which requires Kubernetes 1.34+.

Install Dynamo with the AICR CLI

  1. Install the CLI with Homebrew:
$brew tap NVIDIA/aicr
$brew install aicr

Or use the install script:

$curl -sfL https://raw.githubusercontent.com/NVIDIA/aicr/main/install | bash -s --
  1. Generate a recipe for your environment. Set --platform dynamo and match --service, --accelerator, and --os to your cluster:
$aicr recipe --service eks --accelerator h100 --os ubuntu \
>  --intent inference --platform dynamo -o recipe.yaml
  1. Render the recipe into deployment bundles. Choose the deployer that matches your workflow (helm, argocd, flux, or helmfile):
$aicr bundle --recipe recipe.yaml --deployer helm --output ./bundles
  1. Deploy the generated bundles to your cluster, then validate the running cluster against the recipe:
$aicr validate --recipe recipe.yaml

For manual installation, the full command reference, and the supported environment matrix, see the AICR documentation. Once AICR has installed the Dynamo Platform, continue with the verification steps above and Next Steps below.

Next Steps

Your cluster is ready. Follow the Deployment Overview to choose between applying a tuned DGD recipe, creating a DGD directly, or using DGDR to generate one.

Troubleshooting

“VALIDATION ERROR: Cannot install cluster-wide Dynamo operator”

VALIDATION ERROR: Cannot install cluster-wide Dynamo operator.
Found existing namespace-restricted Dynamo operators in namespaces: ...

Cause: Attempting cluster-wide install on a shared cluster with existing namespace-restricted operators.

Solution: Remove the development/test namespace-restricted operators, then install one cluster-wide operator for production use.

CRDs already exist

Cause: Installing CRDs on a cluster where they’re already present (common on shared clusters).

Solution: The cluster-wide operator’s crd-apply init container manages CRDs automatically. If you encounter conflicts, check existing CRDs with kubectl get crd | grep dynamo.

Pods not starting?

$kubectl describe pod <pod-name> -n $NAMESPACE
$kubectl logs <pod-name> -n $NAMESPACE

Bitnami etcd “unrecognized” image?

$ERROR: Original containers have been substituted for unrecognized ones.

Add to the helm install command:

$--set "etcd.image.repository=bitnamilegacy/etcd" --set "etcd.global.security.allowInsecureImages=true"

Clean uninstall?

$# Uninstall the platform
$helm uninstall dynamo-platform --namespace $NAMESPACE
$
$# List Dynamo CRDs
$kubectl get crd | grep "dynamo.*nvidia.com"
$
$# Delete each CRD
$kubectl delete crd <crd-name>

Reference