Install with AICR

NVIDIA AI Cluster Runtime (AICR) generates one version-locked recipe for the whole GPU and Dynamo stack, renders it into a deployable bundle, and validates the running cluster against the recipe. It is an alternative to the manual steps in Install Dynamo.

This guide is written against AICR v1.1.2, whose Dynamo recipes install the dynamo-platform Helm chart 1.4.2 into the dynamo-system namespace. The commands target Amazon EKS with H100 GPUs and Ubuntu nodes; change the --service, --accelerator, and --os values to match your cluster.

Check for a Dynamo Recipe

AICR covers more environments than Dynamo does. List the combinations that have a Dynamo recipe instead of assuming yours does:

aicr recipe list --intent inference --platform dynamo

Narrow the list with --service, --accelerator, and --os. Each row is a leaf recipe, with two columns that describe it:

• STATUS: the structural verdict (pass, warn, fail, or unknown). pass means the recipe resolves and its charts are pinned. It does not mean the recipe was validated on hardware.
• COVERAGE: the validation checks the recipe declares, in the form R:n D:n P:n C:n for the readiness, deployment, performance, and conformance phases.

If no row matches your cluster, use the manual steps in Install Dynamo.

Prerequisites

• A Kubernetes 1.34 or later cluster, which the Dynamo recipes need for Dynamic Resource Allocation (DRA), with GPU nodes that match the recipe. For this guide, that means EKS GPU node groups running Ubuntu. The Amazon EKS guide creates an EKS Auto Mode cluster, which does not use Ubuntu nodes and does not match this recipe.
• kubectl and Helm access with cluster-admin permissions.
• Image pull access from the GPU nodes to the registries the bundle and your Dynamo runtime images use.
• On EKS, EBS CSI driver permissions through EKS Pod Identity or IRSA. The bundle installs the driver but not its AWS credentials, and without them the Prometheus volume claim stays Pending. See EBS CSI Driver Credentials.
• A cluster without existing installs of components the recipe manages. Run helm list -A and remove or plan around an existing GPU Operator, cert-manager, kube-prometheus-stack, or dynamo-platform release, because the bundle installs its own. On EKS, if the Amazon EBS CSI driver is installed as an EKS add-on, add --set awsebscsidriver:enabled=false to aicr bundle. This also drops the default StorageClass that the bundle’s driver creates, so confirm that kubectl get storageclass shows a (default) class.

brew tap NVIDIA/aicr
brew install aicr

curl -sfL https://get.aicr.run | bash -s --

aicr --version

aicr recipe --service eks --accelerator h100 --os ubuntu \
  --intent inference --platform dynamo -o recipe.yaml

aicr bundle --recipe recipe.yaml --deployer helm \
  --accelerated-node-selector <label>=<value> \
  --output ./bundles

(cd bundles && aicr verify . && chmod +x deploy.sh && ./deploy.sh)

aicr validate --recipe ./bundles/recipe.yaml --phase deployment --phase conformance

kubectl get crd | grep -E "dynamo|grove"
kubectl get pods -n dynamo-system
kubectl get pods -n kai-scheduler

kubectl get nodes -o custom-columns='NAME:.metadata.name,GPUS:.status.allocatable.nvidia\.com/gpu'

Open Network Access on EKS

The Dynamo recipes do not install NATS. The request plane uses TCP and KV events use ZMQ, so frontend and worker pods connect to each other directly. If you place frontend and worker components in different node groups, allow this traffic between their security groups:

The ai-service-metrics conformance check also needs TCP 9090 access to Prometheus from every node group that can run validation jobs, even when frontend and worker share a node group. For the complete rules and an example, see EKS Dynamo Networking Prerequisites.

Disaggregated deployments also transfer KV cache between prefill and decode workers; see EFA on AWS.

Next Steps

Your cluster is ready. Follow the Model Deployment guide to deploy a model with a DynamoGraphDeployment. Use runtime images (vLLM, SGLang, or TensorRT-LLM) that match the installed platform version, which is 1.4.2 for AICR v1.1.2. The bundle owns the platform: change platform settings by regenerating it with aicr bundle --set, and put model and serving settings in your DynamoGraphDeployment.

To upgrade a deployed stack, run aicr upgrade-check and follow Upgrading a Deployed Stack. After an upgrade that changes the platform version, update each DynamoGraphDeployment’s runtime image to match.

To uninstall, run helm uninstall for each component in the reverse order listed under Uninstall in the bundle’s README.md. See Bundle Uninstall. Helm does not delete CRDs. To remove the Dynamo CRDs, see Clean uninstall? in Install Dynamo troubleshooting.

Troubleshooting

aicr recipe fails with requires additional criteria. Run aicr recipe list --intent inference --platform dynamo and use values from a listed row. A partial query such as --os ubuntu --accelerator h100 can be ambiguous, so also set --service.

aicr validate exits with code 2 before running any phase. The readiness pre-flight failed. The output names the failed constraint.

AICR jobs stay Pending. The snapshot agent and the inner validation workloads target GPU nodes. If your GPU nodes carry non-standard labels or taints, pass --node-selector and --toleration to aicr validate. These flags do not affect the validator orchestrator Job.

The ai-service-metrics check fails or requests time out between pods. Security groups can block traffic between node groups. See Open Network Access on EKS.