Kubernetes User Guide
This page covers the Kubernetes path. To run Dynamo directly on a workstation or virtual machine without Kubernetes, see the Local CLI User Guide.
Jump Right In
Install Dynamo and serve a model on a Kubernetes cluster.
Choose a deployment workflow and learn the core Kubernetes resources.
Define and apply a DynamoGraphDeployment for direct control.
Generate a deployment from model and workload requirements.
Browse the Kubernetes Guide
Getting Started
Installation
Install the Dynamo platform and prepare networking, storage, and observability for your cluster.
Install the operator, custom resource definitions, and platform services.
Configure Grove or LeaderWorkerSet for workloads that span nodes.
RDMA setup
Prepare high-speed networking for distributed serving.
Configure InfiniBand for Azure Kubernetes Service.
Configure Elastic Fabric Adapter for Amazon EKS.
Model storage
Choose a storage approach for model weights.
Compare storage options for Azure Kubernetes Service.
Mount Azure Managed Lustre with the Container Storage Interface driver.
Use Amazon Elastic File System for shared model storage.
Managed Kubernetes
Model Deployment
Create and tune Kubernetes deployments for different models and serving patterns.
Choose a deployment workflow and learn the core Kubernetes resources.
Run Dynamo beside a stock inference engine and find backend-specific deployment examples.
Define and apply a DynamoGraphDeployment.
Route requests according to cached prompt prefixes.
Scale prefill and decode workers independently.
Select parallelism and replica settings for a workload.
Deploy a model across multiple Kubernetes nodes.
Reduce model-loading time across pods.
Extend KV cache capacity beyond GPU memory.
Auto deploy with DGDR
Operations
Observe, benchmark, and simulate Kubernetes deployments.
Collect metrics from the operator and serving components.
Measure throughput and latency under load.
Inference simulation with DynoSim