> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# v1.4.1

## Docs

- [Kubernetes User Guide](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/getting-started/introduction.md): Install, deploy, and operate Dynamo on Kubernetes
- [Kubernetes Quickstart](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/getting-started/quickstart.md): Get a model running on Kubernetes and send your first request.
- [Installation Guide](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/installation/install-dynamo.md): Install accelerator support and the Dynamo Platform on Kubernetes.
- [Multinode Orchestration](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/installation/multinode-orchestration.md)
- [Install Gateway API Inference Extension](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/installation/gateway-api-inference-extension.md): Install Gateway API, the Inference Extension, and a compatible Gateway implementation for Dynamo EPP routing.
- [RDMA Setup](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/installation/rdma-setup/overview.md): When Dynamo needs RDMA and where to set it up for your platform
- [RDMA / InfiniBand on AKS](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/installation/rdma-setup/infiniband-on-azure.md)
- [EFA (RDMA over AWS Fabric) on EKS](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/installation/rdma-setup/efa-on-aws.md): Enables GPU-Direct RDMA over AWS Elastic Fabric Adapter so disaggregated workers transfer KV cache across EKS nodes without TCP.
- [Model Storage Overview](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/installation/model-storage/overview.md): Choose Kubernetes storage for model weights, compilation artifacts, and shared caches
- [Storage for Model Caching on AKS](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/installation/model-storage/aks-storage.md)
- [Azure Lustre CSI Driver for AKS](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/installation/model-storage/azure-lustre-csi-driver.md)
- [Amazon EFS Setup for EKS](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/installation/model-storage/efs.md): Creates an Amazon EFS file system so Dynamo workers on EKS share model weights and compilation cache across nodes.
- [Amazon Elastic Kubernetes Service (EKS)](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/installation/managed-kubernetes/eks/eks-setup.md): Deploys the Dynamo platform on Amazon EKS, including cluster creation, GPU node groups, and add-ons.
- [Amazon Elastic Container Service (ECS)](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/installation/managed-kubernetes/eks/ecs.md): Runs Dynamo on Amazon ECS with EC2 GPU clusters for vLLM workers and Fargate tasks for etcd and NATS.
- [Azure Kubernetes Service (AKS)](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/installation/managed-kubernetes/azure/aks-setup.md)
- [AKS Spot VMs](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/installation/managed-kubernetes/azure/spot-vms.md)
- [Google Kubernetes Engine (GKE)](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/installation/managed-kubernetes/gcp/gke-setup.md)
- [Model Deployment](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/model-deployment/introduction.md): Create and configure a DynamoGraphDeployment to serve a model on Kubernetes.
- [Sidecar Backends](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/model-deployment/sidecar-backends-experimental.md): Run Dynamo beside a stock inference engine through its native gRPC API.
- [Deploy with DGD](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/model-deployment/deploy-with-dgd.md): Author a DynamoGraphDeployment spec step by step — choose a model and backend, size parallelism for your hardware, then apply it and send requests.
- [Expose the Frontend](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/model-deployment/expose-the-frontend.md): Route external traffic to a DynamoGraphDeployment's Frontend with a Kubernetes Ingress, a LoadBalancer Service, or the Inference Gateway.
- [Deploy on Intel GPUs with DRA](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/model-deployment/deploy-on-intel-gp-us.md): Adapt a DynamoGraphDeployment for Intel GPUs by building an XPU runtime image and allocating devices with Kubernetes Dynamic Resource Allocation.
- [Multinode Deployments](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/model-deployment/multinode-deployments.md): Scales Dynamo inference across multiple GPU nodes with Grove and KAI-Scheduler for large-model tensor parallelism.
- [Model Caching](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/model-deployment/model-loading/model-caching.md): Download models once and share across all pods in a Kubernetes cluster
- [Fast Model Weight Loading](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/model-deployment/model-loading/fast-model-loading.md): Cut cold-start weight-load time on shared PVC storage with a concurrent loader and correct filesystem layout
- [KV-Aware Routing on Kubernetes](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/kv-aware-routing/overview.md): Choose where Dynamo makes cache-aware worker-selection decisions in a Kubernetes deployment.
- [Using the Dynamo Frontend](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/kv-aware-routing/using-the-dynamo-frontend.md): Configure the Dynamo Frontend to select the worker most likely to have the prompt prefix cached.
- [Using GAIE with Dynamo](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/kv-aware-routing/using-gaie-with-dynamo.md): Add a Dynamo Endpoint Picker Plugin and HTTPRoute to an existing DynamoGraphDeployment.
- [Using GAIE with vanilla vLLM](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/kv-aware-routing/vanilla-vllm-onramp.md): Add Dynamo's advanced KV-aware routing to GAIE deployments that use upstream vLLM images.
- [Disaggregated Serving](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/disaggregated-serving/overview.md): Write a DynamoGraphDeployment that splits prefill and decode into independent worker pools
- [Size a Kubernetes Deployment with AIConfigurator](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/disaggregated-serving/size-with-ai-configurator.md): Generate a tensor- and pipeline-parallel layout for your DGD worker from a latency target
- [Set up KV Cache Offloading](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/kv-cache-offloading/overview.md): Add a KV cache offloading connector to a worker in a DynamoGraphDeployment so KV blocks spill to host memory or disk.
- [Fault Tolerance Overview](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/fault-tolerance/overview.md): Choose request-level and worker-level recovery behaviors for production Dynamo deployments.
- [Request Migration](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/fault-tolerance/request-migration.md): Keep in-flight generations alive across worker failures by retrying them on a healthy worker.
- [Request Rejection](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/fault-tolerance/request-rejection.md): Configure independent load thresholds that shed traffic with HTTP 529 before worker latency or memory use becomes unsafe.
- [Graceful Shutdown](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/fault-tolerance/graceful-shutdown.md): Let workers finish in-flight requests and release resources cleanly when a pod is terminated.
- [Shadow Engine Failover](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/fault-tolerance/shadow-engine-failover.md): Keeps a standby engine attached to GPU-resident model weights so process failures recover without reloading the model.
- [Auto Deployment](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/auto-deployment/overview.md): Generate a DynamoGraphDeployment from model, hardware, workload, and latency requirements.
- [Auto Deploy with DGDR](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/auto-deployment/dgdr-walkthrough.md): Deploy a model by intent — describe the model, workload, and SLA targets, and let Dynamo profile your hardware and generate the DynamoGraphDeployment for you.
- [Dynamo Profiler](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/auto-deployment/dynamo-profiler.md): How the profiler turns a DynamoGraphDeploymentRequest into a sized deployment — the pipeline, the two search modes, and what it optimizes for.
- [Tune the Planner](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/auto-deployment/dynamo-planner.md): Pick the right Planner optimization target and scaling knobs for your workload, and validate the choice before it scales GPUs.
- [Observability](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/operations/observability.md): View Dynamo metrics, configure structured logs and distributed traces, capture scheduler telemetry, and check deployment health
- [Snapshotting GPU Workers](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/operations/cold-start-optimizations/dynamo-snapshot.md): Checkpoints initialized GPU workers with CRIU and cuda-checkpoint so later pods warm-start in seconds instead of minutes.
- [ModelExpress](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/operations/cold-start-optimizations/model-express.md): Speed up model weight distribution across Kubernetes workers
- [DynoSim](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/operations/dynosim/overview.md): Simulate Dynamo deployment choices before spending GPU time
- [DynoSim Simulation Model](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/operations/dynosim/simulation-model.md): Engine behavior, timing sources, KV movement, and fidelity boundaries
- [Simulate a Kubernetes Deployment with Mocker](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/operations/dynosim/live-simulation-with-mocker.md): Exercise the Dynamo frontend, router, and worker lifecycle without GPUs
- [Run a DynoSim Simulation](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/operations/dynosim/simulation-runs.md): Replay a synthetic workload or saved trace against one simulated configuration
- [Sweep DynoSim Configurations](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/operations/dynosim/simulation-sweeps.md): Search simulated topology, worker, and router choices before using GPU time
- [Benchmark Planner Decisions with DynoSim](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/operations/dynosim/planner-simulation-benchmarking.md): Compare topology and scaling decisions against a saved trace
- [Benchmark a Kubernetes Deployment with AIPerf](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/operations/benchmarking-with-ai-perf.md): Measure latency and throughput through a DynamoGraphDeployment frontend
- [Advanced Performance Tuning](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/operations/performance-tuning.md): Measure a deployed configuration and tune the remaining workload-specific parameters
- [LoRA Adapters](https://docs.nvidia.com/dynamo/v1.4.1/kubernetes/operations/lo-ra-adapters.md): Serve fine-tuned LoRA adapters with dynamic loading and routing in Dynamo
- [Local CLI User Guide](https://docs.nvidia.com/dynamo/v1.4.1/cli/getting-started/introduction.md): Install, deploy, and operate Dynamo without Kubernetes
- [Quickstart](https://docs.nvidia.com/dynamo/v1.4.1/cli/getting-started/quickstart.md): Get a Dynamo OpenAI-compatible endpoint running locally on an NVIDIA GPU or Intel XPU.
- [Local Installation](https://docs.nvidia.com/dynamo/v1.4.1/cli/installation/install-dynamo.md): Install Dynamo on a local machine or VM with containers or PyPI
- [Minikube Setup](https://docs.nvidia.com/dynamo/v1.4.1/cli/installation/minikube-setup.md): Sets up a local Minikube cluster with GPU support and Istio for running the Dynamo Kubernetes Platform.
- [Deployment Guide](https://docs.nvidia.com/dynamo/v1.4.1/cli/model-deployment/introduction.md): Deploy and serve a model locally with a Dynamo backend
- [Multi-Node Deployment](https://docs.nvidia.com/dynamo/v1.4.1/cli/model-deployment/multi-node-deployment.md): Span a single model across multiple nodes with the CLI
- [KV-Aware Routing](https://docs.nvidia.com/dynamo/v1.4.1/cli/kv-aware-routing/overview.md): Route requests across workers based on KV cache overlap
- [Standalone Router](https://docs.nvidia.com/dynamo/v1.4.1/cli/kv-aware-routing/standalone-router.md): Run KV-aware routing as its own Dynamo service.
- [Disaggregated Serving](https://docs.nvidia.com/dynamo/v1.4.1/cli/disaggregated-serving/overview.md): Split prefill and decode across independent workers
- [Size a Local Deployment with AIConfigurator](https://docs.nvidia.com/dynamo/v1.4.1/cli/disaggregated-serving/sizing-with-ai-configurator.md): Choose parallelism and worker counts before launching local Dynamo processes
- [Offload KV Cache Locally](https://docs.nvidia.com/dynamo/v1.4.1/cli/kv-cache-offloading/overview.md): Extend KV cache capacity beyond GPU memory when running Dynamo from a container or Python environment
- [Observe a Local Deployment](https://docs.nvidia.com/dynamo/v1.4.1/cli/operations/observability.md): Inspect metrics, traces, logs, request captures, scheduler telemetry, and process health
- [DynoSim](https://docs.nvidia.com/dynamo/v1.4.1/cli/operations/dynosim/overview.md): Simulate Dynamo deployment choices before spending GPU time
- [DynoSim Simulation Model](https://docs.nvidia.com/dynamo/v1.4.1/cli/operations/dynosim/simulation-model.md): Engine behavior, timing sources, KV movement, and fidelity boundaries
- [Simulate a Local Deployment with Mocker](https://docs.nvidia.com/dynamo/v1.4.1/cli/operations/dynosim/mocker-live-simulation.md): Run a GPU-free frontend and simulated worker from the command line
- [Run a DynoSim Simulation](https://docs.nvidia.com/dynamo/v1.4.1/cli/operations/dynosim/simulation-runs.md): Replay a synthetic workload or saved trace against one simulated configuration
- [Sweep DynoSim Configurations](https://docs.nvidia.com/dynamo/v1.4.1/cli/operations/dynosim/simulation-sweeps.md): Search simulated topology, worker, and router choices before using GPU time
- [Benchmark a Local Deployment with AIPerf](https://docs.nvidia.com/dynamo/v1.4.1/cli/operations/benchmarking-with-ai-perf.md): Measure latency and throughput through the local Dynamo frontend
- [LoRA Adapters](https://docs.nvidia.com/dynamo/v1.4.1/cli/operations/lo-ra-adapters.md): Serve fine-tuned LoRA adapters with dynamic loading and routing in Dynamo
- [Agents](https://docs.nvidia.com/dynamo/v1.4.1/agents/overview.md): Agent-aware serving features in Dynamo
- [Session IDs](https://docs.nvidia.com/dynamo/v1.4.1/agents/session-i-ds.md): Identify agent sessions from supported coding agents and custom clients
- [Agent Hints](https://docs.nvidia.com/dynamo/v1.4.1/agents/agent-hints.md): Per-request serving hints for agentic workloads
- [Priority Scheduling](https://docs.nvidia.com/dynamo/v1.4.1/agents/priority-scheduling.md): Request priority across the Dynamo router and backend engines
- [Agent Harnesses](https://docs.nvidia.com/dynamo/v1.4.1/agents/agent-harnesses.md): Point coding-agent CLIs at a Dynamo deployment
- [Agent Tracing](https://docs.nvidia.com/dynamo/v1.4.1/agents/agent-tracing.md): Export Dynamo request traces, tool-call metadata, and Perfetto timelines
- [Agent Trace Replay](https://docs.nvidia.com/dynamo/v1.4.1/agents/agent-simulation.md): Capture an agent workload once, then replay it on real or simulated infrastructure
- [ThunderAgent Program Scheduler](https://docs.nvidia.com/dynamo/v1.4.1/agents/thunder-agent-program-scheduler.md): Program-level scheduling with tool-boundary pause/resume on top of KV-aware routing
- [Reinforcement Learning](https://docs.nvidia.com/dynamo/v1.4.1/reinforcement-learning/overview.md): Use Dynamo as the rollout-serving plane for RL training systems
- [RL Implementation Guide](https://docs.nvidia.com/dynamo/v1.4.1/reinforcement-learning/rl-implementation-guide.md): Connect RL orchestrators to Dynamo inference and engine administration interfaces
- [Multimodal Model Serving](https://docs.nvidia.com/dynamo/v1.4.1/multimodal/overview.md): Deploy multimodal models with image, video, and audio support in Dynamo
- [Parallel Media Decoding](https://docs.nvidia.com/dynamo/v1.4.1/multimodal/parallel-media-decoding.md): Decode image inputs concurrently in the Rust frontend and transfer pixels to inference backends
- [Embedding Cache](https://docs.nvidia.com/dynamo/v1.4.1/multimodal/embedding-cache.md): Cache vision encoder embeddings to skip re-encoding repeated multimodal content
- [Multimodal KV Routing](https://docs.nvidia.com/dynamo/v1.4.1/multimodal/multimodal-kv-routing.md): Include multimodal identity in the router's combined cache-and-load cost
- [Encoder Disaggregation](https://docs.nvidia.com/dynamo/v1.4.1/multimodal/encoder-disaggregation.md): Separate vision encoding into a dedicated worker for independent scaling
- [Video Decode GPU Requirements](https://docs.nvidia.com/dynamo/v1.4.1/multimodal/video-decode-gpu-requirements.md)
- [Additional Media Decoders](https://docs.nvidia.com/dynamo/v1.4.1/multimodal/additional-media-decoders.md): Explicitly install software decoders for input formats the image cannot decode
- [Diffusion](https://docs.nvidia.com/dynamo/v1.4.1/diffusion/overview.md): Deploy diffusion models for image, video, audio, and text generation
- [Text-to-Image](https://docs.nvidia.com/dynamo/v1.4.1/diffusion/text-to-image.md): Generate images from text prompts with vLLM-Omni, SGLang, or TensorRT-LLM
- [Text-to-Video](https://docs.nvidia.com/dynamo/v1.4.1/diffusion/text-to-video.md): Generate videos from text prompts with vLLM-Omni, SGLang, TensorRT-LLM, or FastVideo
- [Image-to-Video](https://docs.nvidia.com/dynamo/v1.4.1/diffusion/image-to-video.md): Animate source images with vLLM-Omni or SGLang
- [Text-to-Audio](https://docs.nvidia.com/dynamo/v1.4.1/diffusion/text-to-audio.md): Synthesize speech with vLLM-Omni through the /v1/audio/speech endpoint
- [Text-to-Text (LLM Diffusion)](https://docs.nvidia.com/dynamo/v1.4.1/diffusion/text-to-text.md): Generate text through iterative refinement with SGLang diffusion language models
- [Overview](https://docs.nvidia.com/dynamo/v1.4.1/parsing/overview.md): How Dynamo turns raw model output into OpenAI-compatible tool_calls and reasoning_content
- [Chat Processors](https://docs.nvidia.com/dynamo/v1.4.1/parsing/chat-processors.md): Pick who parses model output — the dynamo default, or vLLM/SGLang engine fallback
- [Tool Call Parsing (Dynamo)](https://docs.nvidia.com/dynamo/v1.4.1/parsing/tool-call-parsing.md): Connect Dynamo to external tools and services using Dynamo's built-in tool call parsers
- [Structural Tag (Guided Decoding for Tool Calls)](https://docs.nvidia.com/dynamo/v1.4.1/parsing/structural-tag.md): Constrain model output to valid tool call format using xgrammar structural tags
- [Reasoning Parsing (Dynamo)](https://docs.nvidia.com/dynamo/v1.4.1/parsing/reasoning-parsing.md): Configure Dynamo's built-in reasoning parsers for models that emit thinking content
- [Troubleshooting Tool Calls](https://docs.nvidia.com/dynamo/v1.4.1/parsing/troubleshooting-tool-calls.md): Capture raw model output with logprobs so issues can be localized
- [Recipes](https://docs.nvidia.com/dynamo/v1.4.1/recipes/browse.md): Pick a model, choose the hardware/runtime target, and deploy the matching Dynamo recipe.
- [Kimi-K3](https://docs.nvidia.com/dynamo/v1.4.1/recipes/kimi-k3.md): Serve Kimi-K3 with Dynamo and vLLM on GB200 or GB300, aggregated or disaggregated over NVLink.
- [Qwen3.8-2.4T-A95B](https://docs.nvidia.com/dynamo/v1.4.1/recipes/qwen-3-8-2-4t-a95b-fp8.md): Serve Qwen3.8-2.4T-A95B with Dynamo and vLLM or SGLang on GB300 or GB200, aggregated or disaggregated.
- [GLM-5.2](https://docs.nvidia.com/dynamo/v1.4.1/recipes/glm-5-2.md): Serve GLM-5.2 with Dynamo and SGLang on B200 or H200 for long-context agentic workloads.
- [Inkling NVFP4](https://docs.nvidia.com/dynamo/v1.4.1/recipes/inkling.md): Day-0 SGLang recipe for Thinking Machines' Inkling on 8x B200.
- [Kimi-K2.6](https://docs.nvidia.com/dynamo/v1.4.1/recipes/kimi-k2-6.md): Serve moonshotai/Kimi-K2.6 with Dynamo and vLLM, tuned per GPU and workload.
- [Nemotron-3.5-Lightning](https://docs.nvidia.com/dynamo/v1.4.1/recipes/nemotron-3-5-lightning.md): Serve NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 with Dynamo, vLLM, and experimental TensorRT-LLM targets.
- [Nemotron-3-Ultra](https://docs.nvidia.com/dynamo/v1.4.1/recipes/nemotron-3-ultra.md): Serve nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 with Dynamo and vLLM, tuned per GPU and workload.
- [Nemotron-3-Super](https://docs.nvidia.com/dynamo/v1.4.1/recipes/nemotron-3-super.md): Serve NVIDIA-Nemotron-3-Super-120B-A12B with Dynamo and vLLM, tuned per GPU and workload.
- [GLM-5 NVFP4](https://docs.nvidia.com/dynamo/v1.4.1/recipes/glm-5-nvfp4.md): Serve GLM-5 NVFP4 with Dynamo and SGLang, disaggregated across five GB200 nodes.
- [DeepSeek-V4-Pro](https://docs.nvidia.com/dynamo/v1.4.1/recipes/deepseek-v4-pro.md): Serve DeepSeek-V4-Pro with Dynamo and vLLM — aggregated or disaggregated, on B200 or H200, tuned for the agentic workload.
- [DeepSeek-V4-Flash](https://docs.nvidia.com/dynamo/v1.4.1/recipes/deepseek-v4-flash.md): Serve DeepSeek-V4-Flash with Dynamo and vLLM — aggregated or disaggregated, on B200 or H200, tuned for the agentic workload.
- [DeepSeek V3.2 NVFP4](https://docs.nvidia.com/dynamo/v1.4.1/recipes/deepseek-v3-2-nvfp4.md): Serve nvidia/DeepSeek-V3.2-NVFP4 with Dynamo and TensorRT-LLM, disaggregated with KV routing and WideEP on GB200 NVL72.
- [GPT-OSS-120B](https://docs.nvidia.com/dynamo/v1.4.1/recipes/gpt-oss-120b.md): Serve openai/gpt-oss-120b with Dynamo on vLLM (B200/H200) or TensorRT-LLM (GB200), aggregated or disaggregated.
- [Qwen3-32B](https://docs.nvidia.com/dynamo/v1.4.1/recipes/qwen3-32b.md): Serve Qwen3-32B with Dynamo and vLLM — disaggregated serving with KV-aware routing for multi-turn conversational traffic.
- [Qwen3.5-122B-A10B](https://docs.nvidia.com/dynamo/v1.4.1/recipes/qwen3-5-122b.md): Serve Qwen3.5-122B-A10B with Dynamo and vLLM on B200 or H200 for long-context agentic workloads.
- [Qwen3-235B-A22B FP8](https://docs.nvidia.com/dynamo/v1.4.1/recipes/qwen3-235b-a22b-fp8.md): Serve Qwen/Qwen3-235B-A22B-FP8 with Dynamo and TensorRT-LLM on 16 Hopper or Blackwell GPUs.
- [Qwen3-32B FP8](https://docs.nvidia.com/dynamo/v1.4.1/recipes/qwen3-32b-fp8.md): Serve Qwen/Qwen3-32B-FP8 with Dynamo on TensorRT-LLM or vLLM, aggregated or disaggregated.
- [Llama-3.3-70B FP8](https://docs.nvidia.com/dynamo/v1.4.1/recipes/llama-3-3-70b.md): Serve Llama-3.3-70B FP8 with Dynamo and vLLM, sized from one node to two.
- [vLLM Deployment Templates](https://docs.nvidia.com/dynamo/v1.4.1/recipes/kubernetes-templates/dgd/v-llm.md): Ready-to-apply DynamoGraphDeployment manifests for serving vLLM with Dynamo on Kubernetes.
- [TensorRT-LLM Deployment Templates](https://docs.nvidia.com/dynamo/v1.4.1/recipes/kubernetes-templates/dgd/tensor-rt-llm.md): Ready-to-apply DynamoGraphDeployment manifests for serving TensorRT-LLM with Dynamo on Kubernetes.
- [SGLang Deployment Templates](https://docs.nvidia.com/dynamo/v1.4.1/recipes/kubernetes-templates/dgd/sg-lang.md): Ready-to-apply DynamoGraphDeployment manifests for serving SGLang with Dynamo on Kubernetes.
- [DGDR Templates](https://docs.nvidia.com/dynamo/v1.4.1/recipes/kubernetes-templates/dgdr.md): Ready-to-apply DynamoGraphDeploymentRequest manifests for profiling and generating DynamoGraphDeployments.
- [vLLM Local Deployment Examples](https://docs.nvidia.com/dynamo/v1.4.1/recipes/cli-templates/v-llm.md): Copyable commands for running the vLLM launch scripts in the Dynamo repository.
- [TensorRT-LLM Local Deployment Examples](https://docs.nvidia.com/dynamo/v1.4.1/recipes/cli-templates/tensor-rt-llm.md): Copyable commands for running the TensorRT-LLM launch scripts in the Dynamo repository.
- [SGLang Local Deployment Examples](https://docs.nvidia.com/dynamo/v1.4.1/recipes/cli-templates/sg-lang.md): Copyable commands for running the SGLang launch scripts in the Dynamo repository.
- [Feature Benchmarks](https://docs.nvidia.com/dynamo/v1.4.1/recipes/benchmarks.md): Start with the feature or performance claim you want to inspect.
- [Overall Architecture](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/overview.md): Architecture and components of the Dynamo inference runtime
- [Architecture Flow](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/concepts/system-architecture/architecture-flow.md)
- [Distributed Runtime](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/concepts/system-architecture/distributed-runtime.md)
- [Sidecar Backends](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/concepts/system-architecture/sidecar-backends-experimental.md): Run Dynamo beside a stock inference engine through its native gRPC API.
- [Disaggregated Serving](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/concepts/system-architecture/disaggregated-serving.md)
- [Discovery Plane](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/concepts/communication-planes/discovery-plane.md)
- [Request Plane](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/concepts/communication-planes/request-plane.md)
- [Event Plane](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/concepts/communication-planes/event-plane.md)
- [Observability Architecture](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/concepts/observability-architecture.md): Signal transport, request correlation, and active worker health checks
- [DynoSim Architecture](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/concepts/simulation/dyno-sim-architecture.md): How the replay harness composes simulated engines, routing, and Planner behavior
- [Request Migration Architecture](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/concepts/fault-tolerance/request-migration-architecture.md): Preserve token state and reissue in-flight requests on healthy workers after a worker failure.
- [Request Cancellation Architecture](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/concepts/fault-tolerance/request-cancellation-architecture.md): Cancel in-flight work across the Frontend and workers when a client disconnects.
- [Request Rejection Architecture](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/concepts/fault-tolerance/request-rejection-architecture.md): Worker-load event processing, busy-state aggregation, overload errors, and hard worker admission limits.
- [Graceful Shutdown Architecture](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/concepts/fault-tolerance/graceful-shutdown-architecture.md)
- [Fault Tolerance Testing](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/concepts/fault-tolerance/fault-tolerance-testing.md): Pytest infrastructure for cancellation, migration, etcd high availability, and hardware fault injection.
- [Frontend](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/frontend/overview.md): API gateway that exposes OpenAI HTTP and KServe gRPC endpoints and handles request preprocessing, routing, and response formatting.
- [Frontend Guide](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/frontend/frontend-guide.md): Configures the KServe gRPC frontend, including supported endpoints, gRPC tuning, and registering OpenAI or tensor-based backends.
- [Python Route Extensions](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/frontend/python-route-extensions.md): Add trusted HTTP routes to the Dynamo Frontend from an installed Python package or importable module.
- [Tokenizer](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/frontend/tokenizer.md): Selects between the default HuggingFace and fastokens tokenizer backends for BPE models served through the Dynamo Frontend.
- [Router](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/router/overview.md): KV cache-aware router that picks workers by combined prefill and decode cost to maximize throughput and minimize latency.
- [Router Guide](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/router/router-guide.md): Deployment modes, quick start, and page map for Dynamo routing docs
- [Routing Concepts](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/router/routing-concepts.md): Cost model, worker selection, and routing primitives for the Dynamo router
- [Router Filtering](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/router/router-filtering.md): Candidate eligibility, DP-rank filtering, and queue backpressure boundaries
- [Configuration and Tuning](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/router/configuration-and-tuning.md): Router behavior, event transport, load tracking, and tuning guidance
- [Deficit Round Robin Queue Scheduling](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/router/deficit-round-robin.md): Weighted arbitration across router policy classes
- [Disaggregated Serving](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/router/disaggregated-serving.md): Prefill and decode routing with the Dynamo router
- [Offloading Support Matrix](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/router/offloading-support-matrix.md): Per-framework support and setup for KV routing with tiered KV cache offloading
- [Topology-Aware KV Transfer](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/router/topology-aware-kv-transfer.md): Runtime metadata and decode routing semantics for topology-aware prefill/decode handoff
- [Multi-DC KV Routing and the DC Relay](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/router/multi-dc-kv-routing.md): Aggregate exact KV ownership in each DC and publish compact state to a global consumer
- [Router Operations](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/router/router-operations.md): Replica topology, state management, recovery, and remote indexers
- [Router Testing](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/router/router-testing.md): Test layers for router changes
- [Router Examples](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/router/router-examples.md): Worked examples for the KvRouter Python API, Kubernetes deployments, and custom routing patterns.
- [Global Router Configuration](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/router/global-router-configuration.md): Field reference for the CLI flags and JSON config file consumed by the Global Router service.
- [Standalone KV Indexer](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/router/standalone-indexer.md): Run the KV cache indexer as an independent HTTP service for querying block state
- [Standalone Slot Tracker](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/router/standalone-slot-tracker.md): Run active-request load accounting as an independent HTTP service
- [Standalone Selection Service](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/router/standalone-selection.md): Select workers and account for reservations without forwarding inference requests
- [KV Event Replay — Dynamo vs vLLM](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/router/kv-event-replay-comparison.md): How the two systems handle gap detection, replay, and recovery for KV cache events
- [Router Design](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/router/router-design.md)
- [Planner](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/planner/overview.md): Autoscaler that adjusts prefill and decode replicas using engine performance models and traffic prediction to meet TTFT and ITL SLAs.
- [Planner Guide](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/planner/planner-guide.md): Configures Planner optimization targets, scaling modes, and PlannerConfig fields for production deployments.
- [Global Planner Deployment Guide](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/planner/global-planner-guide.md): Deploys GlobalPlanner as the centralized scaling layer for multi-DGD policy enforcement and single-endpoint multi-pool deployments.
- [Planner Examples](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/planner/planner-examples.md): Examples for custom load predictors and the VirtualConnector for non-Kubernetes scaling environments.
- [Planner Design](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/planner/planner-design.md)
- [Profiler](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/profiler/overview.md): Automated performance analysis that finds optimal prefill and decode parallelism and generates SLA-driven deployment configurations.
- [Profiler Guide](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/profiler/profiler-guide.md): Runs the AI Configurator-driven profiling pipeline that turns a model, SLA targets, and backend into a ready-to-deploy DGD.
- [Profiler Examples](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/profiler/profiler-examples.md): Complete DGDR profiling examples for dense and MoE models across rapid and thorough search strategies.
- [AI Simulate (Experimental)](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/ai-simulate/overview.md): Experimental simulation and configuration-search tools for Dynamo deployments
- [Spica](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/ai-simulate/spica/overview.md): Experimental replay-backed configuration search for Dynamo deployments
- [Spica Search Flow](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/ai-simulate/spica/search-flow.md): How Spica turns a deployment-tuning problem into a replay-backed search
- [Spica Optimization Goals](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/ai-simulate/spica/optimization-goals.md): Objective metrics, SLA constraints, and Pareto-front scoring
- [Spica Traffic](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/ai-simulate/spica/traffic.md): Trace, request-rate, concurrency, and KV-load workload definitions
- [Spica Search Space](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/ai-simulate/spica/search-space.md): Searchable deployment, engine, router, Planner, and parallelism settings
- [Spica Unrolled Samples](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/ai-simulate/spica/unrolled-samples.md): How Spica turns a search trial into a concrete deployment configuration
- [GLM-5-FP8 Disaggregated Pareto Sweep](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/ai-simulate/spica/glm-5-fp-8-pareto-sweep.md): Experimental Spica search for a GLM-5-FP8 Pareto frontier on B200 GPUs
- [Planner Goodput-per-GPU Sweep](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/ai-simulate/spica/planner-goodput-per-gpu-sweep.md): Experimental Spica search over Planner policies for MiniMax-M2.5 on B200 GPUs
- [Router End-to-End Latency Sweep](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/ai-simulate/spica/router-end-to-end-latency-sweep.md): Experimental Spica search over KV router settings for MiniMax-M2.5 on B200 GPUs
- [KVBM](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/kvbm/overview.md): Unified memory layer and write-through KV cache spanning GPU, host, SSD, and remote storage via NIXL for TensorRT-LLM and vLLM.
- [KVBM Guide](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/kvbm/kvbm-guide.md): Enable KV offloading using KV Block Manager (KVBM) for Dynamo deployments
- [KVBM Design](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/kvbm/kvbm-design.md)
- [SGLang](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/sg-lang/overview.md): SGLang engines run in Dynamo's distributed runtime with disaggregated serving, KV-aware routing, and request cancellation.
- [SGLang Sidecar](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/sg-lang/sidecar.md): Run Dynamo beside a stock SGLang engine through native gRPC.
- [Reference Guide](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/sg-lang/reference-guide.md): Architecture, configuration, and operational details for the SGLang backend
- [SGLang for Agentic Workloads](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/sg-lang/agents-on-sg-lang.md): Priority scheduling and session control for multi-turn agentic serving
- [SGLang Chat Processor](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/sg-lang/chat-processor.md): SGLang-native preprocessing and postprocessing for chat completions
- [Disaggregation](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/sg-lang/disaggregation.md): SGLang runs prefill and decode as separate workers in Dynamo, coordinated by a bootstrap handshake with RDMA-based KV cache transfer.
- [Using HiCache](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/sg-lang/hi-cache.md): Hierarchical KV caching with tier-aware router integration
- [Observability](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/sg-lang/observability.md): Prometheus metrics, forward pass telemetry, distributed tracing, and Grafana dashboards for SGLang workers in Dynamo.
- [SGLang Multimodal](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/sg-lang/sglang-multimodal.md)
- [TensorRT-LLM](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/tensor-rt-llm/overview.md): TensorRT-LLM engines run in Dynamo's distributed runtime with disaggregated serving, KV-aware routing, and multinode support.
- [TensorRT-LLM Sidecar](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/tensor-rt-llm/sidecar.md): Run Dynamo beside a stock TensorRT-LLM engine through native gRPC.
- [Reference Guide](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/tensor-rt-llm/reference-guide.md): Features, configuration, and operational details for the TensorRT-LLM backend
- [TensorRT-LLM Observability](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/tensor-rt-llm/observability.md): TensorRT-LLM exposes request, latency, and KV cache transfer metrics through Dynamo's Prometheus /metrics endpoint.
- [Known Issues and Mitigations](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/tensor-rt-llm/known-issues.md): Known issues for TensorRT-LLM in Dynamo, including disaggregated KV cache exhaustion and driver mismatch errors, with mitigations.
- [TensorRT-LLM Multimodal](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/tensor-rt-llm/tensorrt-llm-multimodal.md)
- [vLLM](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/v-llm/overview.md)
- [vLLM Sidecar](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/v-llm/sidecar.md): Run Dynamo beside a stock vLLM engine through native gRPC.
- [Reference Guide](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/v-llm/reference-guide.md): Configuration, arguments, and operational details for the vLLM backend
- [vLLM Chat Processor](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/v-llm/chat-processor.md): vLLM-native preprocessing and postprocessing for chat completions
- [KV Cache Offloading](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/v-llm/kv-cache-offloading.md): CPU and disk offloading integrations for vLLM in Dynamo
- [Native KV Offloading](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/v-llm/native-kv-offloading.md): vLLM's OffloadingConnector with KV-aware routing in Dynamo
- [vLLM Observability](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/v-llm/observability.md): vLLM exposes request, latency, scheduler, and disaggregation metrics through Dynamo's Prometheus /metrics endpoint.
- [vLLM Multimodal](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/v-llm/vllm-multimodal.md)
- [vLLM-Omni Disaggregated Multi-Stage Serving](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/v-llm/v-llm-omni-disaggregated-serving.md): Architecture of multi-stage vLLM-Omni pipelines (e.g. AR + Diffusion), with each stage on its own GPU for independent scaling and isolation.
- [Mocker Engine Architecture](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/modular-components/backends/mocker/mocker-engine-architecture.md)
- [Dynamo Operator](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/kubernetes/kubernetes-operator/dynamo-operator.md): Reference for the Dynamo Kubernetes operator covering its controllers, deployment modes, and reconciliation workflow.
- [Service Discovery](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/kubernetes/kubernetes-operator/service-discovery.md): Lets Dynamo frontends, workers, and planners locate each other on Kubernetes via native CRDs and EndpointSlices, or via etcd.
- [Webhooks](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/kubernetes/kubernetes-operator/webhooks.md): Dynamo operator admission webhooks that validate and default Dynamo custom resources before persistence.
- [Managing Models with DynamoModel](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/kubernetes/kubernetes-operator/managing-models-dynamo-model.md): Deploys LoRA adapters and tracks model endpoints with the DynamoModel custom resource alongside DGDs.
- [Autoscaling](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/kubernetes/kubernetes-operator/autoscaling.md): Scales DGD services with the DynamoGraphDeploymentScalingAdapter using KEDA, Kubernetes HPA, or the Dynamo Planner.
- [Rolling Updates](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/kubernetes/kubernetes-operator/rolling-update.md): Updates DGD worker images, resources, and arguments with managed rolling updates across Deployment, Grove, and LWS backends.
- [Disaggregated Inference Communication Guide](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/kubernetes/kubernetes-operator/disagg-communication.md): Best practices for prefill/decode worker communication on Kubernetes
- [Gateway API Routing Architecture](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/kubernetes/kubernetes-operator/gateway-api-routing.md): How Gateway API, the Dynamo Endpoint Picker Plugin, and worker Frontend sidecars divide request-routing responsibilities.
- [Grove](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/kubernetes/multinode/grove.md): Kubernetes API for orchestrating disaggregated inference with gang scheduling, multi-level autoscaling, and topology-aware placement.
- [Topology Aware Scheduling](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/kubernetes/multinode/topology-aware-scheduling.md): Packs prefill, decode, and routing pods within the same rack or zone to reduce inter-node latency.
- [Topology-Aware KV Transfer](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/kubernetes/multinode/topology-aware-kv-transfer.md): Keep disaggregated prefill and decode KV-cache transfers within a selected topology domain
- [LWS](https://docs.nvidia.com/dynamo/v1.4.1/knowledge-base/kubernetes/multinode/lws.md): LeaderWorkerSet integration for multinode Dynamo deployments
- [Building from Source](https://docs.nvidia.com/dynamo/v1.4.1/advanced-customizations/building-from-source.md): Build Dynamo from source for development and contributions
- [Custom Vision Encoders](https://docs.nvidia.com/dynamo/v1.4.1/advanced-customizations/custom-vision-encoders.md): Run a bespoke vision tower in an aggregated Dynamo vLLM worker
- [FastTokens Tokenizer](https://docs.nvidia.com/dynamo/v1.4.1/advanced-customizations/fastokens-tokenizer.md): Reduce frontend tokenization latency for long-context BPE models
- [Custom Backend Overview](https://docs.nvidia.com/dynamo/v1.4.1/advanced-customizations/writing-custom-backends/custom-backend-overview.md): Choose the right path for bringing your own engine to Dynamo
- [Writing Unified Backends](https://docs.nvidia.com/dynamo/v1.4.1/advanced-customizations/writing-custom-backends/writing-unified-backends.md): Choose Python or Rust for Dynamo's shared backend contract
- [Writing Python Workers in Dynamo](https://docs.nvidia.com/dynamo/v1.4.1/advanced-customizations/writing-custom-backends/writing-python-workers.md): Create custom Python workers and engines for Dynamo
- [Publish KV Events from a Custom Backend](https://docs.nvidia.com/dynamo/v1.4.1/advanced-customizations/writing-custom-backends/publish-kv-events.md): Report cache allocation and eviction events to Dynamo's KV router
- [Runtime Containers](https://docs.nvidia.com/dynamo/v1.4.1/advanced-customizations/writing-custom-backends/runtime-containers.md): Build Dynamo runtime images for built-in or custom backends
- [Metrics Developer Guide](https://docs.nvidia.com/dynamo/v1.4.1/advanced-customizations/metrics-developer-guide.md): Reference for creating counters, gauges, and histograms with Dynamo's metrics API, surfaced on the local /metrics endpoint.
- [Developing the Operator with Tilt](https://docs.nvidia.com/dynamo/v1.4.1/advanced-customizations/developing-with-tilt.md): Fast, live-reload development loop for the Dynamo Kubernetes operator
- [Compatibility](https://docs.nvidia.com/dynamo/v1.4.1/reference/compatibility.md): Hardware, platform, feature, and mixed-version support for Dynamo
- [Releases](https://docs.nvidia.com/dynamo/v1.4.1/reference/releases.md): Release history and per-release notes for every Dynamo release, mirrored from GitHub
- [Dynamo v1.5.0](https://docs.nvidia.com/dynamo/v1.4.1/reference/releases/v1-5-0.md): Release notes for Dynamo v1.5.0 (GA Sep 18, 2026), including patch release v1.5.1
- [Dynamo v1.4.0](https://docs.nvidia.com/dynamo/v1.4.1/reference/releases/v1-4-0.md): Release notes for Dynamo v1.4.0 (GA Aug 14, 2026), including patch releases v1.4.1 and v1.4.2
- [Dynamo v1.3.0](https://docs.nvidia.com/dynamo/v1.4.1/reference/releases/v1-3-0.md): Release notes for Dynamo v1.3.0 (GA Jul 20, 2026), including patch release v1.3.1
- [Dynamo v1.2.0](https://docs.nvidia.com/dynamo/v1.4.1/reference/releases/v1-2-0.md): Release notes for Dynamo v1.2.0 (GA Jun 2, 2026), including patch release v1.2.1
- [Dynamo v1.1.0](https://docs.nvidia.com/dynamo/v1.4.1/reference/releases/v1-1-0.md): Release notes for Dynamo v1.1.0 (GA May 1, 2026), including patch release v1.1.1
- [Dynamo v1.0.0](https://docs.nvidia.com/dynamo/v1.4.1/reference/releases/v1-0-0.md): Release notes for Dynamo v1.0.0 (GA Mar 12, 2026), including patch releases v1.0.1, v1.0.2
- [Known Issues](https://docs.nvidia.com/dynamo/v1.4.1/reference/releases/known-issues.md): Known issues per Dynamo release, mirrored from the GitHub release notes
- [Deprecations](https://docs.nvidia.com/dynamo/v1.4.1/reference/releases/deprecations.md): Breaking changes, deprecations, and removals across Dynamo releases
- [Release Artifacts](https://docs.nvidia.com/dynamo/v1.4.1/reference/release-artifacts.md): Container images, Python wheels, Helm charts, and Rust crates for the current release, plus early access builds and release history.
- [Nightly Releases](https://docs.nvidia.com/dynamo/v1.4.1/reference/nightly-releases.md): Nightly container images, Python wheels, install patterns, and current backend versions.
- [Model Early Access Builds](https://docs.nvidia.com/dynamo/v1.4.1/reference/model-early-access-builds.md): Per-model early access container builds shipped ahead of stable releases
- [Examples](https://docs.nvidia.com/dynamo/v1.4.1/reference/examples.md): Reference deployments for SGLang, TensorRT-LLM, vLLM, and custom Dynamo backends.
- [Glossary](https://docs.nvidia.com/dynamo/v1.4.1/reference/glossary.md): Definitions of core Dynamo terms covering disaggregated serving, KV cache, components, and the distributed runtime.
- [API Reference](https://docs.nvidia.com/dynamo/v1.4.1/reference/api.md): Language-by-language public surface for Dynamo, discovered from the source tree
- [Python API](https://docs.nvidia.com/dynamo/v1.4.1/reference/api/python.md): Public Python surfaces on the Dynamo distributed runtime, statically discovered from the source tree.
- [dynamo._core](https://docs.nvidia.com/dynamo/v1.4.1/reference/api/python/_core.md): Rust-backed distributed runtime, KV router, and endpoint bindings.
- [dynamo.runtime](https://docs.nvidia.com/dynamo/v1.4.1/reference/api/python/runtime.md): Decorators and re-exports for defining Dynamo workers and endpoints.
- [dynamo.llm](https://docs.nvidia.com/dynamo/v1.4.1/reference/api/python/llm.md): High-level LLM primitives for building request pipelines.
- [dynamo.frontend](https://docs.nvidia.com/dynamo/v1.4.1/reference/api/python/frontend.md): OpenAI-compatible HTTP frontend, argument parsing, and pre/post-processing.
- [dynamo.common](https://docs.nvidia.com/dynamo/v1.4.1/reference/api/python/common.md): Shared configuration groups, storage adapters, and utility helpers.
- [dynamo.health_check](https://docs.nvidia.com/dynamo/v1.4.1/reference/api/python/health_check.md): Health-check payload types and environment-driven configuration.
- [dynamo.logits_processing](https://docs.nvidia.com/dynamo/v1.4.1/reference/api/python/logits_processing.md): Custom logits processors for LLM token generation.
- [dynamo.planner](https://docs.nvidia.com/dynamo/v1.4.1/reference/api/python/planner.md): Scaling connectors, decision types, and Planner configuration.
- [dynamo.router](https://docs.nvidia.com/dynamo/v1.4.1/reference/api/python/router.md): Request-router configuration and command-line argument groups.
- [dynamo.mocker](https://docs.nvidia.com/dynamo/v1.4.1/reference/api/python/mocker.md): Mock engine used to run Dynamo workflows without a real GPU backend.
- [dynamo.nixl_connect](https://docs.nvidia.com/dynamo/v1.4.1/reference/api/python/nixl_connect.md): NIXL RDMA connector primitives for KV cache and tensor transport.
- [Rust API](https://docs.nvidia.com/dynamo/v1.4.1/reference/api/rust.md): Published Dynamo crates with release-pinned docs.rs links.
- [DynamoGraphDeployment (DGD) Reference](https://docs.nvidia.com/dynamo/v1.4.1/reference/api/kubernetes/dynamo-graph-deployment.md): Field reference for the DynamoGraphDeployment custom resource — the canonical, live description of a Dynamo inference graph.
- [DynamoGraphDeploymentRequest (DGDR) Reference](https://docs.nvidia.com/dynamo/v1.4.1/reference/api/kubernetes/dynamo-graph-deployment-request.md): Field reference and lifecycle for the DynamoGraphDeploymentRequest custom resource — Dynamo's deploy-by-intent generator.
- [DynamoComponentDeployment (DCD) Reference](https://docs.nvidia.com/dynamo/v1.4.1/reference/api/kubernetes/dynamo-component-deployment.md): Field reference for the DynamoComponentDeployment custom resource — the single Dynamo component that a DynamoGraphDeployment reconciles into pods.
- [API Reference](https://docs.nvidia.com/dynamo/v1.4.1/reference/api/kubernetes/full-api-reference.md): Full Kubernetes CRD + operator-config API reference, generated from the operator Go types.
- [Runtime Configuration (DynamoRuntimeConfig)](https://docs.nvidia.com/dynamo/v1.4.1/reference/components/runtime-configuration.md): Field reference for the cross-cutting Dynamo runtime CLI flags and environment variables shared by every backend and the frontend.
- [Frontend Configuration Reference](https://docs.nvidia.com/dynamo/v1.4.1/reference/components/frontend-configuration.md): Field reference for the Dynamo Frontend CLI arguments, environment variables, and HTTP endpoints.
- [Planner Configuration (PlannerConfig)](https://docs.nvidia.com/dynamo/v1.4.1/reference/components/planner-configuration.md): Field reference for the PlannerConfig JSON/YAML object consumed by the Dynamo Planner service.
- [Profiler Configuration Reference](https://docs.nvidia.com/dynamo/v1.4.1/reference/components/profiler-configuration.md): Field reference for the Dynamo profiler CLI arguments — the SLA profiler entry point and the low-level per-endpoint profiler.
- [KVBM Configuration Reference](https://docs.nvidia.com/dynamo/v1.4.1/reference/components/kvbm-configuration.md): Field reference for the KV Block Manager environment variables and the layered KVBM_* / TOML configuration system.
- [Mocker CLI Reference](https://docs.nvidia.com/dynamo/v1.4.1/reference/components/mocker-cli-reference.md): Command-line flags for the python -m dynamo.mocker simulated worker.
- [DynoSim Replay CLI Reference](https://docs.nvidia.com/dynamo/v1.4.1/reference/components/dyno-sim-replay-cli-reference.md): Flags, engine-args JSON fields, constraints, and report schema for python -m dynamo.replay.
- [DynoSim Sweep Reference](https://docs.nvidia.com/dynamo/v1.4.1/reference/components/dyno-sim-sweep-reference.md): Input models, search behavior, and result fields for replay_optimize
- [Global Router Configuration](https://docs.nvidia.com/dynamo/v1.4.1/reference/components/global-router-configuration.md): Field reference for the CLI flags and JSON config file consumed by the Global Router service.
- [Gateway API Routing Reference](https://docs.nvidia.com/dynamo/v1.4.1/reference/components/gateway-api-routing.md): Dynamo Endpoint Picker Plugin fields, generated resources, runtime settings, and request contracts.
- [ThunderAgent Scheduler Reference](https://docs.nvidia.com/dynamo/v1.4.1/reference/components/thunder-agent-scheduler.md): Control-loop flags, constraints, capacity behavior, and scheduler log records
- [vLLM Configuration (DynamoVllmConfig)](https://docs.nvidia.com/dynamo/v1.4.1/reference/backends/v-llm-configuration.md): Field reference for the Dynamo-specific CLI flags and environment variables of the vLLM backend wrapper.
- [vLLM-Omni Configuration](https://docs.nvidia.com/dynamo/v1.4.1/reference/backends/v-llm-omni-configuration.md): CLI flag reference for the vLLM-Omni multimodal backend (python -m dynamo.vllm.omni).
- [SGLang Configuration (DynamoSGLangConfig)](https://docs.nvidia.com/dynamo/v1.4.1/reference/backends/sg-lang-configuration.md): Field reference for the Dynamo-specific CLI flags and environment variables of the SGLang backend wrapper.
- [TensorRT-LLM Configuration (DynamoTrtllmConfig)](https://docs.nvidia.com/dynamo/v1.4.1/reference/backends/tensor-rt-llm-configuration.md): Field reference for the Dynamo-specific CLI flags and environment variables of the TensorRT-LLM backend wrapper.
- [Local Observability Stack Reference](https://docs.nvidia.com/dynamo/v1.4.1/reference/observability/local-stack.md): Services, ports, signal flow, and configuration files for the Docker Compose stack
- [Health Check Reference](https://docs.nvidia.com/dynamo/v1.4.1/reference/observability/health-checks.md): Health endpoints, status codes, paths, and active-check configuration
- [Logging Reference](https://docs.nvidia.com/dynamo/v1.4.1/reference/observability/logging.md): Log formats, levels, correlation fields, and backend controls
- [Forward Pass Metrics Trace Reference](https://docs.nvidia.com/dynamo/v1.4.1/reference/observability/forward-pass-metrics-traces.md): Configuration, topology support, file rotation, and the dynamo.fpm.trace.v1 schema
- [Request Trace Reference](https://docs.nvidia.com/dynamo/v1.4.1/reference/observability/request-traces.md): Record selection, sinks, rotation, OTLP export, and the dynamo.request.trace.v1 schema
- [Environment Variables](https://docs.nvidia.com/dynamo/v1.4.1/reference/observability/environment-variables.md): Every environment variable that controls Dynamo metrics, tracing, logging, health checks, and request capture.
- [Metrics Catalog](https://docs.nvidia.com/dynamo/v1.4.1/reference/observability/metrics-catalog.md): Every dynamo_* metric emitted by the frontend, backend workers, and router — name, type, and meaning.
- [Engine Metrics Comparison](https://docs.nvidia.com/dynamo/v1.4.1/reference/observability/metrics-comparison.md)
- [Metric Labels](https://docs.nvidia.com/dynamo/v1.4.1/reference/observability/metric-labels.md): Every label dimension attached to Dynamo metric series — where each one comes from and the values it can take.
- [Operator Metrics](https://docs.nvidia.com/dynamo/v1.4.1/reference/observability/operator-metrics.md): Kubernetes-only dynamo_operator_* metrics and the annotations and Helm values that enable metrics collection.
- [Local Resource Monitor (Local)](https://docs.nvidia.com/dynamo/v1.4.1/reference/observability/local-resource-monitor-local.md): Host-side per-process VRAM / PCIe / CPU exporter for engine-startup profiling — metrics and CLI flags.
- [NIXL Connect API](https://docs.nvidia.com/dynamo/v1.4.1/reference/nixl-connect/overview.md): Python library for dynamic, RDMA-accelerated data transfers between workers in a Dynamo graph.
- [External publications](https://docs.nvidia.com/dynamo/v1.4.1/external-publications.md): Articles, deep dives, and deployment write-ups about NVIDIA Dynamo published by customers and partners.
- [Research publications](https://docs.nvidia.com/dynamo/v1.4.1/research-publications.md): Papers that use, extend, or benchmark NVIDIA Dynamo, from the teams building on it and the wider systems community.
- [Dynamo Blog](https://docs.nvidia.com/dynamo/v1.4.1/digest.md): Engineering deep dives, architecture notes, and project updates from the NVIDIA Dynamo team.
- [DynoSim: Simulating the Pareto Frontier](https://docs.nvidia.com/dynamo/v1.4.1/digest/dynosim-pareto-frontier.md): DynoSim is a workload-driven discrete-event simulation of the NVIDIA Dynamo serving stack for mapping Pareto frontiers before real-cluster validation.
- [NVIDIA Dynamo Snapshot: Fast Startup for Inference Workloads on Kubernetes](https://docs.nvidia.com/dynamo/v1.4.1/digest/dynamo-snapshot-fast-startup.md): NVIDIA Dynamo Snapshot combines CUDA and host checkpointing to restore warm inference workers quickly on Kubernetes.
- [Dynamo Day 0 support for TokenSpeed](https://docs.nvidia.com/dynamo/v1.4.1/digest/tokenspeed-day-0.md): Dynamo adds day-0 TokenSpeed support with the Dynamo frontend for Kimi K2.5.
- [Streaming Tokens and Tools: Multi-Turn Agentic Harness Support in Dynamo](https://docs.nvidia.com/dynamo/v1.4.1/digest/agentic-harnesses.md): Streaming Tokens and Tools: Multi-Turn Agentic Harness Support in Dynamo
- [Full-Stack Optimizations for Agentic Inference with Dynamo](https://docs.nvidia.com/dynamo/v1.4.1/digest/agentic-inference.md): How Dynamo optimizes for agentic workloads at three layers: frontend API, router, and KV cache management.
- [Flash Indexer: A Story of Inter-Galactic KV Routing](https://docs.nvidia.com/dynamo/v1.4.1/digest/flash-indexer.md): Dynamo's Flash Indexer tracks every cached KV block across all inference workers at 170M ops/s. Six iterations of data structure design got it there.
- [Community](https://docs.nvidia.com/dynamo/v1.4.1/community.md): Community meetings, events, and project channels
- [Contributing to Dynamo](https://docs.nvidia.com/dynamo/v1.4.1/contributing/contribution-guide.md): Choose a contribution path and find the guide for your next step
- [Contributor Getting Started](https://docs.nvidia.com/dynamo/v1.4.1/contributing/getting-started.md): Set up a fork and create a signed, validated change
- [Contribution Flow](https://docs.nvidia.com/dynamo/v1.4.1/contributing/contribution-flow.md): Take a change from proposal through pull request review
- [Code Quality](https://docs.nvidia.com/dynamo/v1.4.1/contributing/code-quality.md): Style, testing, validation, and commit conventions for Dynamo changes
- [DCO and Licensing](https://docs.nvidia.com/dynamo/v1.4.1/contributing/dco-and-licensing.md): Sign, verify, and repair commits for contribution acceptance
- [Documentation Style Guide](https://docs.nvidia.com/dynamo/v1.4.1/contributing/documentation/style-guide.md): How to write and structure Dynamo docs, examples, and recipes
- [How to Build and Publish Dynamo Docs](https://docs.nvidia.com/dynamo/v1.4.1/contributing/documentation/building-and-publishing.md): Branch model, sync workflow, and publishing pipeline for the Fern-powered Dynamo documentation site.


> **Note:** This page contains both a page directory (above) and the landing page content (below). The page directory is generated for agent use and does not appear on the landing page.

> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Dynamo

![](/dynamo/_fern-files/dynamo.docs.buildwithfern.com/4019188fbddfe63ae7a9ac420a19191e951921157d63b773d2ae3b61ff03728e/assets/img/dynamo-logo.svg)