> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Kubernetes User Guide

This page covers the **Kubernetes** path. To run Dynamo directly on a workstation or virtual machine without Kubernetes, see the [Local CLI User Guide](/dynamo/dev/cli/getting-started/introduction).

## Jump Right In

#### [Quickstart](/dynamo/dev/kubernetes/getting-started/quickstart)

Install Dynamo and serve a model on a Kubernetes cluster.

#### [Model Deployment Overview](/dynamo/dev/kubernetes/model-deployment/introduction)

Choose a deployment workflow and learn the core Kubernetes resources.

#### [Deploy with DGD](/dynamo/dev/kubernetes/model-deployment/deploy-with-dgd)

Define and apply a DynamoGraphDeployment for direct control.

#### [Auto Deploy with DGDR](/dynamo/dev/kubernetes/auto-deployment/dgdr-walkthrough)

Generate a deployment from model and workload requirements.

## Browse the Kubernetes Guide

#### Getting Started

#### [Quickstart](/dynamo/dev/kubernetes/getting-started/quickstart)

Install Dynamo and serve a model on a Kubernetes cluster.

#### [Compatibility](/dynamo/dev/reference/compatibility)

Check supported platforms, backends, features, and dependencies.

#### Installation

Install the Dynamo platform and prepare networking, storage, and observability for your cluster.

#### [Install Dynamo](/dynamo/dev/kubernetes/installation/install-dynamo)

Install the operator, custom resource definitions, and platform services.

#### [Multinode Orchestration](/dynamo/dev/kubernetes/installation/multinode-orchestration)

Configure Grove or LeaderWorkerSet for workloads that span nodes.

#### [Observability](/dynamo/dev/kubernetes/installation/observability)

Set up the monitoring stack for a Kubernetes deployment.

**RDMA setup**

#### [RDMA Overview](/dynamo/dev/kubernetes/installation/rdma-setup/overview)

Prepare high-speed networking for distributed serving.

#### [InfiniBand on Azure](/dynamo/dev/kubernetes/installation/rdma-setup/infiniband-on-azure)

Configure InfiniBand for Azure Kubernetes Service.

#### [EFA on AWS](/dynamo/dev/kubernetes/installation/rdma-setup/efa-on-aws)

Configure Elastic Fabric Adapter for Amazon EKS.

**Model storage**

#### [Model Storage Overview](/dynamo/dev/kubernetes/installation/model-storage/overview)

Choose a storage approach for model weights.

#### [AKS Storage](/dynamo/dev/kubernetes/installation/model-storage/aks-storage)

Compare storage options for Azure Kubernetes Service.

#### [Azure Lustre CSI Driver](/dynamo/dev/kubernetes/installation/model-storage/azure-lustre-csi-driver)

Mount Azure Managed Lustre with the Container Storage Interface driver.

#### [EFS](/dynamo/dev/kubernetes/installation/model-storage/efs)

Use Amazon Elastic File System for shared model storage.

**Managed Kubernetes**

#### [EKS Setup](/dynamo/dev/kubernetes/installation/managed-kubernetes/eks/eks-setup)

Prepare an Amazon EKS cluster for Dynamo.

#### [ECS](/dynamo/dev/kubernetes/installation/managed-kubernetes/eks/ecs)

Deploy Dynamo with Amazon Elastic Container Service.

#### [AKS Setup](/dynamo/dev/kubernetes/installation/managed-kubernetes/azure/aks-setup)

Prepare an Azure Kubernetes Service cluster for Dynamo.

#### [Spot VMs](/dynamo/dev/kubernetes/installation/managed-kubernetes/azure/spot-vms)

Use interruptible capacity for suitable workloads.

#### [GKE Setup](/dynamo/dev/kubernetes/installation/managed-kubernetes/gcp/gke-setup)

Prepare a Google Kubernetes Engine cluster for Dynamo.

#### Model Deployment

Create and tune Kubernetes deployments for different models and serving patterns.

#### [Model Deployment Introduction](/dynamo/dev/kubernetes/model-deployment/introduction)

Choose a deployment workflow and learn the core Kubernetes resources.

#### [Deploy with DGD](/dynamo/dev/kubernetes/model-deployment/deploy-with-dgd)

Define and apply a DynamoGraphDeployment.

#### [KV-Aware Routing](/dynamo/dev/kubernetes/model-deployment/kv-aware-routing/dynamo-frontend)

Route requests according to cached prompt prefixes.

#### [Disaggregated Serving](/dynamo/dev/kubernetes/model-deployment/disaggregated-serving)

Scale prefill and decode workers independently.

#### [Sizing with AIConfigurator](/dynamo/dev/kubernetes/model-deployment/size-with-ai-configurator)

Select parallelism and replica settings for a workload.

#### [Multinode Deployments](/dynamo/dev/kubernetes/model-deployment/multinode-deployments)

Deploy a model across multiple Kubernetes nodes.

#### [Model Caching](/dynamo/dev/kubernetes/model-deployment/model-caching)

Reduce model-loading time across pods.

#### [KV Cache Offloading](/dynamo/dev/kubernetes/model-deployment/kv-cache-offloading)

Extend KV cache capacity beyond GPU memory.

**Auto deploy with DGDR**

#### [Auto Deploy with DGDR](/dynamo/dev/kubernetes/auto-deployment/dgdr-walkthrough)

Generate a deployment from model and workload requirements.

#### [Dynamo Profiler](/dynamo/dev/kubernetes/auto-deployment/dynamo-profiler)

Profile candidate configurations on your hardware.

#### [Dynamo Planner](/dynamo/dev/kubernetes/auto-deployment/dynamo-planner)

Select and adjust deployment configurations.

#### Operations

Observe, benchmark, and simulate Kubernetes deployments.

#### [Observability](/dynamo/dev/kubernetes/operations/observability)

Collect metrics from the operator and serving components.

#### [Benchmarking with AIPerf](/dynamo/dev/kubernetes/operations/benchmarking-with-ai-perf)

Measure throughput and latency under load.

**Inference simulation with DynoSim**

#### [DynoSim Overview](/dynamo/dev/kubernetes/operations/dynosim/overview)

Learn the simulation workflow and components.

#### [Simulate a Kubernetes Deployment](/dynamo/dev/kubernetes/operations/dynosim/live-simulation-with-mocker)

Exercise the Kubernetes frontend and router with simulated workers.