> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# v1.1.1

## Docs

- [Quickstart](https://docs.nvidia.com/dynamo/v1.1.1/getting-started/quickstart.md): Get a Dynamo OpenAI-compatible endpoint running in a container in about 5 minutes.
- [Introduction to Dynamo](https://docs.nvidia.com/dynamo/v1.1.1/getting-started/introduction.md)
- [Local Installation](https://docs.nvidia.com/dynamo/v1.1.1/getting-started/local-installation.md): Install and run Dynamo on a local machine or VM with containers or PyPI
- [Building from Source](https://docs.nvidia.com/dynamo/v1.1.1/getting-started/building-from-source.md): Build Dynamo from source for development and contributions
- [Quickstart](https://docs.nvidia.com/dynamo/v1.1.1/getting-started/kubernetes-deployment.md)
- [Contribution Guide](https://docs.nvidia.com/dynamo/v1.1.1/getting-started/contribution-guide.md): How to contribute to Dynamo
- [Support Matrix](https://docs.nvidia.com/dynamo/v1.1.1/resources/support-matrix.md): Hardware, software, and build compatibility for Dynamo
- [Feature Matrix](https://docs.nvidia.com/dynamo/v1.1.1/resources/feature-matrix.md)
- [Release Artifacts](https://docs.nvidia.com/dynamo/v1.1.1/resources/release-artifacts.md): Container images, Python wheels, Helm charts, Rust crates, and release history
- [Examples](https://docs.nvidia.com/dynamo/v1.1.1/resources/examples.md)
- [Glossary](https://docs.nvidia.com/dynamo/v1.1.1/resources/glossary.md)
- [Dynamo Blog](https://docs.nvidia.com/dynamo/v1.1.1/digest.md): Engineering deep dives, architecture notes, and project updates from the NVIDIA Dynamo team.
- [Dynamo Day 0 support for TokenSpeed](https://docs.nvidia.com/dynamo/v1.1.1/digest/tokenspeed-day-0.md): Dynamo adds day-0 TokenSpeed support with the Dynamo frontend for Kimi K2.5.
- [Streaming Tokens and Tools: Multi-Turn Agentic Harness Support in Dynamo](https://docs.nvidia.com/dynamo/v1.1.1/digest/agentic-harnesses.md): Streaming Tokens and Tools: Multi-Turn Agentic Harness Support in Dynamo
- [Full-Stack Optimizations for Agentic Inference with Dynamo](https://docs.nvidia.com/dynamo/v1.1.1/digest/agentic-inference.md): How Dynamo optimizes for agentic workloads at three layers: frontend API, router, and KV cache management.
- [Flash Indexer: A Story of Inter-Galactic KV Routing](https://docs.nvidia.com/dynamo/v1.1.1/digest/flash-indexer.md): Dynamo's Flash Indexer tracks every cached KV block across all inference workers at 170M ops/s. Six iterations of data structure design got it there.
- [Installation Guide](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/deployment-guide/installation-guide.md)
- [Model Deployment Guide](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/deployment-guide/model-deployment-guide.md)
- [DGDR Reference](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/deployment-guide/dgdr-reference.md)
- [Dynamo Operator](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/deployment-guide/dynamo-operator.md)
- [Service Discovery](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/deployment-guide/service-discovery.md)
- [Webhooks](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/deployment-guide/webhooks.md)
- [Minikube Setup](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/deployment-guide/minikube-setup.md)
- [Managing Models with DynamoModel](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/deployment-guide/managing-models-with-dynamo-model.md)
- [Autoscaling](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/deployment-guide/autoscaling.md)
- [Rolling Updates](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/deployment-guide/rolling-update.md)
- [Developing the Operator with Tilt](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/deployment-guide/developing-with-tilt.md): Fast, live-reload development loop for the Dynamo Kubernetes operator
- [Inference Gateway (GAIE)](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/deployment-guide/inference-gateway-gaie.md)
- [Snapshot](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/deployment-guide/snapshot.md)
- [Shadow Engine Failover](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/deployment-guide/shadow-engine-failover.md)
- [Disaggregated Inference Communication Guide](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/deployment-guide/disagg-communication.md): Best practices for prefill/decode worker communication on Kubernetes
- [Metrics](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/observability-k-8-s/metrics.md)
- [Logging](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/observability-k-8-s/logging.md)
- [Operator Metrics](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/observability-k-8-s/operator-metrics.md)
- [Multinode Deployments](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/multinode/multinode-deployments.md)
- [Grove](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/multinode/grove.md)
- [Topology Aware Scheduling](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/multinode/topology-aware-scheduling.md)
- [Amazon Elastic Kubernetes Service (EKS)](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/cloud-provider-guides/aws/eks-setup.md)
- [Amazon EFS Setup for EKS](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/cloud-provider-guides/aws/efs.md)
- [Amazon Elastic Container Service (ECS)](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/cloud-provider-guides/aws/ecs.md)
- [Azure Kubernetes Service (AKS)](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/cloud-provider-guides/azure/aks-setup.md)
- [RDMA / InfiniBand on AKS](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/cloud-provider-guides/azure/rdma-infini-band.md)
- [Storage for Model Caching on AKS](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/cloud-provider-guides/azure/aks-storage.md)
- [Azure Lustre CSI Driver for AKS](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/cloud-provider-guides/azure/azure-lustre-csi-driver.md)
- [AKS Spot VMs](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/cloud-provider-guides/azure/spot-v-ms.md)
- [Google Kubernetes Engine (GKE)](https://docs.nvidia.com/dynamo/v1.1.1/kubernetes-deployment/cloud-provider-guides/gcp/gke-setup.md)
- [Disaggregated Serving](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/disaggregated-serving.md): Find optimal prefill/decode configuration for disaggregated serving deployments
- [Router Guide](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/kv-cache-aware-routing.md): Deployment modes, quick start, and page map for Dynamo routing docs
- [KVBM Guide](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/kv-cache-offloading.md): Enable KV offloading using KV Block Manager (KVBM) for Dynamo deployments
- [Tool Calling](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/tool-calling.md): Parse tool calls from model output and surface them as OpenAI-compatible tool_calls
- [Tool Call Parsing (Dynamo)](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/tool-calling/tool-call-parsing-dynamo.md): Connect Dynamo to external tools and services using Dynamo's built-in tool call parsers
- [Tool Call Parsing (Engine Fallback)](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/tool-calling/tool-call-parsing-engine-fallback.md): Use upstream vLLM or SGLang tool-call parsers when Dynamo does not ship one
- [Troubleshooting Tool Calls](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/tool-calling/troubleshooting-tool-calls.md): Capture raw model output with logprobs so issues can be localized
- [Reasoning](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/reasoning.md): Separate reasoning content from assistant output for chain-of-thought models
- [Reasoning Parsing (Dynamo)](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/reasoning/reasoning-parsing-dynamo.md): Configure Dynamo's built-in reasoning parsers for models that emit thinking content
- [Reasoning Parsing (Engine Fallback)](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/reasoning/reasoning-parsing-engine-fallback.md): Use upstream vLLM or SGLang reasoning parsers when Dynamo does not ship one
- [Agents](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/agents.md): Agent-aware serving features in Dynamo
- [Agent Tracing](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/agents/agent-tracing.md): Attach trajectory identity and export Dynamo request and tool-event telemetry
- [Agent Hints](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/agents/agent-hints.md): Per-request serving hints for agentic workloads
- [SGLang for Agentic Workloads](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/agents/sg-lang-for-agentic-workloads.md): Priority scheduling and session control for multi-turn agentic serving
- [Multimodal Model Serving](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/multimodal.md): Deploy multimodal models with image, video, and audio support in Dynamo
- [Embedding Cache](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/multimodal/embedding-cache.md): Cache vision encoder embeddings to skip re-encoding repeated images
- [Encoder Disaggregation](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/multimodal/encoder-disaggregation.md): Separate vision encoding into a dedicated worker for independent scaling
- [Multimodal KV Routing](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/multimodal/multimodal-kv-routing.md): Route multimodal requests to workers with the best KV cache overlap
- [Diffusion](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/diffusion.md): Deploy diffusion models for text-to-image, text-to-video, and more in Dynamo
- [FastVideo](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/diffusion/fastvideo.md)
- [vLLM-Omni](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/diffusion/v-llm-omni.md)
- [Diffusion](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/diffusion/sg-lang-diffusion.md)
- [Video Diffusion Support (Experimental)](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/diffusion/trt-llm-diffusion.md)
- [LoRA Adapters](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/lo-ra-adapters.md): Serve fine-tuned LoRA adapters with dynamic loading and routing in Dynamo
- [Observability (Local)](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/observability-local.md): Monitor Dynamo deployments with metrics, logging, and tracing
- [Prometheus + Grafana Setup](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/observability-local/prometheus-grafana-setup.md)
- [Metrics](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/observability-local/metrics.md)
- [Metrics Developer Guide](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/observability-local/metrics-developer-guide.md)
- [Health Checks](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/observability-local/health-checks.md)
- [Tracing](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/observability-local/tracing.md)
- [Logging](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/observability-local/logging.md)
- [Fault Tolerance](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/fault-tolerance.md): Handle failures gracefully with request migration, cancellation, and graceful shutdown
- [Request Migration](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/fault-tolerance/request-migration.md)
- [Request Cancellation](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/fault-tolerance/request-cancellation.md)
- [Request Rejection](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/fault-tolerance/request-rejection.md)
- [Graceful Shutdown](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/fault-tolerance/graceful-shutdown.md)
- [Testing](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/fault-tolerance/testing.md)
- [Dynamo Benchmarking](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/benchmarking.md): Benchmark and compare performance across Dynamo deployment configurations
- [Mocker](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/mocker/testing-with-mocker.md)
- [Mocker Trace Replay](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/mocker/mocker-offline-trace-replay.md): Replay Mooncake-style traces through the mocker in offline or online mode
- [Writing Python Workers in Dynamo](https://docs.nvidia.com/dynamo/v1.1.1/user-guides/writing-python-workers.md): Create custom Python workers and engines for Dynamo
- [SGLang](https://docs.nvidia.com/dynamo/v1.1.1/backends/sg-lang.md)
- [Reference Guide](https://docs.nvidia.com/dynamo/v1.1.1/backends/sg-lang/reference-guide.md): Architecture, configuration, and operational details for the SGLang backend
- [SGLang Chat Processor](https://docs.nvidia.com/dynamo/v1.1.1/backends/sg-lang/chat-processor.md): SGLang-native preprocessing and postprocessing for chat completions
- [Examples](https://docs.nvidia.com/dynamo/v1.1.1/backends/sg-lang/examples.md)
- [Disaggregation](https://docs.nvidia.com/dynamo/v1.1.1/backends/sg-lang/disaggregation.md)
- [Observability](https://docs.nvidia.com/dynamo/v1.1.1/backends/sg-lang/observability.md)
- [TensorRT-LLM](https://docs.nvidia.com/dynamo/v1.1.1/backends/tensor-rt-llm.md)
- [Reference Guide](https://docs.nvidia.com/dynamo/v1.1.1/backends/tensor-rt-llm/reference-guide.md): Features, configuration, and operational details for the TensorRT-LLM backend
- [Examples](https://docs.nvidia.com/dynamo/v1.1.1/backends/tensor-rt-llm/examples.md)
- [Prometheus](https://docs.nvidia.com/dynamo/v1.1.1/backends/tensor-rt-llm/observability.md)
- [Known Issues and Mitigations](https://docs.nvidia.com/dynamo/v1.1.1/backends/tensor-rt-llm/known-issues-and-mitigations.md)
- [vLLM](https://docs.nvidia.com/dynamo/v1.1.1/backends/v-llm.md)
- [Reference Guide](https://docs.nvidia.com/dynamo/v1.1.1/backends/v-llm/reference-guide.md): Configuration, arguments, and operational details for the vLLM backend
- [vLLM Chat Processor](https://docs.nvidia.com/dynamo/v1.1.1/backends/v-llm/frontend-processor-fallback.md): vLLM-native preprocessing and postprocessing for chat completions
- [Examples](https://docs.nvidia.com/dynamo/v1.1.1/backends/v-llm/examples.md)
- [KV Cache Offloading](https://docs.nvidia.com/dynamo/v1.1.1/backends/v-llm/kv-cache-offloading.md): CPU and disk offloading integrations for vLLM in Dynamo
- [Prometheus](https://docs.nvidia.com/dynamo/v1.1.1/backends/v-llm/observability.md)
- [Frontend](https://docs.nvidia.com/dynamo/v1.1.1/components/frontend.md)
- [Frontend Guide](https://docs.nvidia.com/dynamo/v1.1.1/components/frontend/frontend-guide.md)
- [Tokenizer](https://docs.nvidia.com/dynamo/v1.1.1/components/frontend/tokenizer.md)
- [Router](https://docs.nvidia.com/dynamo/v1.1.1/components/router.md)
- [Routing Concepts](https://docs.nvidia.com/dynamo/v1.1.1/components/router/routing-concepts.md): Cost model, worker selection, and routing primitives for the Dynamo router
- [Configuration and Tuning](https://docs.nvidia.com/dynamo/v1.1.1/components/router/configuration-and-tuning.md): Router flags, event transport, load tracking, and tuning guidance
- [Disaggregated Serving](https://docs.nvidia.com/dynamo/v1.1.1/components/router/disaggregated-serving.md): Prefill and decode routing with the Dynamo router
- [Router Operations](https://docs.nvidia.com/dynamo/v1.1.1/components/router/router-operations.md): Replica topology, remote indexers, state management, and recovery
- [Router Examples](https://docs.nvidia.com/dynamo/v1.1.1/components/router/router-examples.md)
- [Standalone KV Indexer](https://docs.nvidia.com/dynamo/v1.1.1/components/router/standalone-indexer.md): Run the KV cache indexer as an independent HTTP service for querying block state
- [KV Event Replay — Dynamo vs vLLM](https://docs.nvidia.com/dynamo/v1.1.1/components/router/kv-event-replay-dynamo-vs-v-llm.md): How the two systems handle gap detection, replay, and recovery for KV cache events
- [Planner](https://docs.nvidia.com/dynamo/v1.1.1/components/planner.md)
- [Planner Guide](https://docs.nvidia.com/dynamo/v1.1.1/components/planner/planner-guide.md)
- [Planner Examples](https://docs.nvidia.com/dynamo/v1.1.1/components/planner/planner-examples.md)
- [Profiler](https://docs.nvidia.com/dynamo/v1.1.1/components/profiler.md)
- [Profiler Guide](https://docs.nvidia.com/dynamo/v1.1.1/components/profiler/profiler-guide.md)
- [Profiler Examples](https://docs.nvidia.com/dynamo/v1.1.1/components/profiler/profiler-examples.md)
- [KVBM](https://docs.nvidia.com/dynamo/v1.1.1/components/kvbm.md)
- [LMCache](https://docs.nvidia.com/dynamo/v1.1.1/integrations/lm-cache.md)
- [HiCache](https://docs.nvidia.com/dynamo/v1.1.1/integrations/sg-lang-hi-cache.md): Hierarchical KV caching with tier-aware router integration
- [FlexKV](https://docs.nvidia.com/dynamo/v1.1.1/integrations/flex-kv.md)
- [KV Events for Custom Engines](https://docs.nvidia.com/dynamo/v1.1.1/integrations/kv-events-for-custom-engines.md)
- [Overall Architecture](https://docs.nvidia.com/dynamo/v1.1.1/design-docs/overall-architecture.md): Architecture and components of the Dynamo inference runtime
- [Architecture Flow](https://docs.nvidia.com/dynamo/v1.1.1/design-docs/architecture-flow.md)
- [Disaggregated Serving](https://docs.nvidia.com/dynamo/v1.1.1/design-docs/disaggregated-serving.md)
- [Distributed Runtime](https://docs.nvidia.com/dynamo/v1.1.1/design-docs/distributed-runtime.md)
- [Discovery Plane](https://docs.nvidia.com/dynamo/v1.1.1/design-docs/communication-planes/discovery-plane.md)
- [Request Plane](https://docs.nvidia.com/dynamo/v1.1.1/design-docs/communication-planes/request-plane.md)
- [Event Plane](https://docs.nvidia.com/dynamo/v1.1.1/design-docs/communication-planes/event-plane.md)
- [Router Design](https://docs.nvidia.com/dynamo/v1.1.1/design-docs/component-design/router-design.md)
- [KVBM Design](https://docs.nvidia.com/dynamo/v1.1.1/design-docs/component-design/kvbm-design.md)
- [Planner Design](https://docs.nvidia.com/dynamo/v1.1.1/design-docs/component-design/planner-design.md)
- [How to Build and Publish Dynamo Docs](https://docs.nvidia.com/dynamo/v1.1.1/documentation/dynamo-docs-guide.md)