> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# v1.5.1

## Docs

- [Kubernetes User Guide](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/getting-started/introduction.md): Install, deploy, and operate Dynamo on Kubernetes
- [Kubernetes Quickstart](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/getting-started/quickstart.md): Get a model running on Kubernetes and send your first request.
- [Installation Guide](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/installation/install-dynamo.md): Install accelerator support and the Dynamo Platform on Kubernetes.
- [Multinode Orchestration](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/installation/multinode-orchestration.md)
- [Install Gateway API Inference Extension](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/installation/gateway-api-inference-extension.md): Install Gateway API, the Inference Extension, and a compatible Gateway implementation for Dynamo EPP routing.
- [Operator TLS](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/installation/operator-tls.md): Configure TLS once at the platform level and auto-inject it into every deployment
- [RDMA Setup](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/installation/rdma-setup/overview.md): When Dynamo needs RDMA and where to set it up for your platform
- [RDMA / InfiniBand on AKS](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/installation/rdma-setup/infiniband-on-azure.md)
- [EFA (RDMA over AWS Fabric) on EKS](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/installation/rdma-setup/efa-on-aws.md): Enables GPU-Direct RDMA over AWS Elastic Fabric Adapter so disaggregated workers transfer KV cache across EKS nodes without TCP.
- [Model Storage Overview](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/installation/model-storage/overview.md): Choose Kubernetes storage for model weights, compilation artifacts, and shared caches
- [Storage for Model Caching on AKS](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/installation/model-storage/aks-storage.md)
- [Azure Lustre CSI Driver for AKS](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/installation/model-storage/azure-lustre-csi-driver.md)
- [Amazon EFS Setup for EKS](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/installation/model-storage/efs.md): Creates an Amazon EFS file system so Dynamo workers on EKS share model weights and compilation cache across nodes.
- [Amazon Elastic Kubernetes Service (EKS)](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/installation/managed-kubernetes/eks/eks-setup.md): Deploys the Dynamo platform on Amazon EKS, including cluster creation, GPU node groups, and add-ons.
- [Amazon Elastic Container Service (ECS)](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/installation/managed-kubernetes/eks/ecs.md): Runs Dynamo on Amazon ECS with EC2 GPU clusters for vLLM workers and Fargate tasks for etcd and NATS.
- [Azure Kubernetes Service (AKS)](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/installation/managed-kubernetes/azure/aks-setup.md)
- [AKS Spot VMs](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/installation/managed-kubernetes/azure/spot-vms.md)
- [Google Kubernetes Engine (GKE)](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/installation/managed-kubernetes/gcp/gke-setup.md)
- [Model Deployment](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/model-deployment/introduction.md): Create and configure a DynamoGraphDeployment to serve a model on Kubernetes.
- [Sidecar Backends](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/model-deployment/sidecar-backends-experimental.md): Run Dynamo beside a stock inference engine through its native gRPC API.
- [Deploy with DGD](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/model-deployment/deploy-with-dgd.md): Author a DynamoGraphDeployment spec step by step — choose a model and backend, size parallelism for your hardware, then apply it and send requests.
- [Expose the Frontend](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/model-deployment/expose-the-frontend.md): Route external traffic to a DynamoGraphDeployment's Frontend with a Kubernetes Ingress, a LoadBalancer Service, or the Inference Gateway.
- [Deploy on Intel GPUs with DRA](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/model-deployment/deploy-on-intel-gp-us.md): Adapt a DynamoGraphDeployment for Intel GPUs by building an XPU runtime image and allocating devices with Kubernetes Dynamic Resource Allocation.
- [Fast Model Weight Loading](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/model-deployment/model-loading/fast-model-loading.md): Cut cold-start weight-load time on shared PVC storage with a concurrent loader and correct filesystem layout
- [Load Models from MatrixHub](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/model-deployment/model-loading/matrix-hub.md): Configure Dynamo on Kubernetes to retrieve model weights from an in-cluster MatrixHub registry.
- [KV-Aware Routing on Kubernetes](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/kv-aware-routing/overview.md): Choose where Dynamo makes cache-aware worker-selection decisions in a Kubernetes deployment.
- [Using the Dynamo Frontend](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/kv-aware-routing/using-the-dynamo-frontend.md): Configure the Dynamo Frontend to select the worker most likely to have the prompt prefix cached.
- [Using GAIE with Dynamo](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/kv-aware-routing/using-gaie-with-dynamo.md): Add a Dynamo Endpoint Picker Plugin and HTTPRoute to an existing DynamoGraphDeployment.
- [Using GAIE with vanilla vLLM](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/kv-aware-routing/vanilla-vllm-onramp.md): Add Dynamo's advanced KV-aware routing to GAIE deployments that use upstream vLLM images.
- [Disaggregated Serving](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/disaggregated-serving/overview.md): Write a DynamoGraphDeployment that splits prefill and decode into independent worker pools
- [Size a Kubernetes Deployment with AIConfigurator](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/disaggregated-serving/size-with-ai-configurator.md): Generate a tensor- and pipeline-parallel layout for your DGD worker from a latency target
- [Set up KV Cache Offloading](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/kv-cache-offloading/overview.md): Add a KV cache offloading connector to a worker in a DynamoGraphDeployment so KV blocks spill to host memory or disk.
- [Deploy LMCache MP](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/kv-cache-offloading/deploy-lm-cache-mp.md): Run aggregated vLLM serving with KV cache offloaded to a per-node LMCache MP DaemonSet over cross-Pod CUDA IPC.
- [Fault Tolerance Overview](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/fault-tolerance/overview.md): Choose request-level and worker-level recovery behaviors for production Dynamo deployments.
- [Request Migration](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/fault-tolerance/request-migration.md): Keep in-flight generations alive across worker failures and shutdown interruptions by retrying them on a healthy worker.
- [Request Rejection](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/fault-tolerance/request-rejection.md): Configure independent load thresholds that shed traffic with HTTP 529 before worker latency or memory use becomes unsafe.
- [Graceful Shutdown](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/fault-tolerance/graceful-shutdown.md): Let workers finish in-flight requests and release resources cleanly when a pod is terminated.
- [Auto Deployment](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/auto-deployment/overview.md): Generate a DynamoGraphDeployment from model, hardware, workload, and latency requirements.
- [Auto Deploy with DGDR](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/auto-deployment/dgdr-walkthrough.md): Deploy a model by intent — describe the model, workload, and SLA targets, and let Dynamo profile your hardware and generate the DynamoGraphDeployment for you.
- [Dynamo Profiler](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/auto-deployment/dynamo-profiler.md): How the profiler turns a DynamoGraphDeploymentRequest into a sized deployment — the pipeline, the two search modes, and what it optimizes for.
- [Tune the Planner](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/auto-deployment/dynamo-planner.md): Pick the right Planner optimization target and scaling knobs for your workload, and validate the choice before it scales GPUs.
- [Observability](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/operations/observability.md): View Dynamo metrics, configure structured logs and distributed traces, capture scheduler telemetry, and check deployment health
- [Benchmark a Kubernetes Deployment with AIPerf](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/operations/benchmarking-with-ai-perf.md): Measure latency and throughput through a DynamoGraphDeployment frontend
- [Advanced Performance Tuning](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/operations/performance-tuning.md): Measure a deployed configuration and tune the remaining workload-specific parameters
- [LoRA Adapters](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/kubernetes/operations/lo-ra-adapters.md): Serve fine-tuned LoRA adapters with dynamic loading and routing in Dynamo
- [Local Guide](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/cli/getting-started/introduction.md): Install, deploy, and operate Dynamo without Kubernetes
- [Quickstart](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/cli/getting-started/quickstart.md): Get a Dynamo OpenAI-compatible endpoint running locally on an NVIDIA GPU or Intel XPU.
- [Local Installation](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/cli/installation/install-dynamo.md): Install Dynamo on a local machine or VM with containers or PyPI
- [Minikube Setup](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/cli/installation/minikube-setup.md): Sets up a local Minikube cluster with GPU support and Istio for running the Dynamo Kubernetes Platform.
- [Install Observability](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/cli/installation/observability.md): Install the local Prometheus, Grafana, Tempo, and Loki stack with Docker Compose
- [Deployment Guide](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/cli/model-deployment/introduction.md): Deploy and serve a model locally with a Dynamo backend
- [KV-Aware Routing](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/cli/kv-aware-routing/overview.md): Route requests across workers based on KV cache overlap
- [Standalone Router](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/cli/kv-aware-routing/standalone-router.md): Run KV-aware routing as its own Dynamo service.
- [Disaggregated Serving](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/cli/disaggregated-serving/overview.md): Split prefill and decode across independent workers
- [Size a Local Deployment with AIConfigurator](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/cli/disaggregated-serving/sizing-with-ai-configurator.md): Choose parallelism and worker counts before launching local Dynamo processes
- [Offload KV Cache Locally](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/cli/kv-cache-offloading/overview.md): Extend KV cache capacity beyond GPU memory when running Dynamo from a container or Python environment
- [Observe a Local Deployment](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/cli/operations/observability.md): Inspect metrics, traces, logs, request captures, scheduler telemetry, and process health
- [Benchmark a Local Deployment with AIPerf](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/cli/operations/benchmarking-with-ai-perf.md): Measure latency and throughput through the local Dynamo frontend
- [LoRA Adapters](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/cli/operations/lo-ra-adapters.md): Serve fine-tuned LoRA adapters with dynamic loading and routing in Dynamo
- [Overall Architecture](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/overview.md): Architecture and components of the Dynamo inference runtime
- [Architecture](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/architecture.md): Request flow, distributed runtime, service discovery, and communication planes
- [Disaggregated Serving](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/system-architecture/disaggregated-serving.md)
- [KV-Aware Routing](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/system-architecture/kv-aware-routing.md): Select workers using reusable KV cache state and projected active load.
- [Observability Architecture](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/observability-architecture.md): Signal transport, request correlation, and active worker health checks
- [DynoSim](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/overview.md): Simulate Dynamo deployment choices before spending GPU time
- [DynoSim Architecture](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/dyno-sim-architecture.md): How the replay harness composes simulated engines, routing, and Planner behavior
- [DynoSim Simulation Model](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/simulation-model.md): Engine behavior, timing sources, KV movement, and fidelity boundaries
- [Run a DynoSim Simulation](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/simulation-runs.md): Predict a synthetic workload or saved trace against one simulated configuration
- [Sweep DynoSim Configurations](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/simulation-sweeps.md): Recommend simulated topology, worker, and router choices before using GPU time
- [Simulate a Local Deployment with Mocker](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/local-mocker.md): Run a GPU-free frontend and simulated worker from the command line
- [Simulate a Kubernetes Deployment with Mocker](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/kubernetes-mocker.md): Exercise the Dynamo frontend, router, and worker lifecycle without GPUs
- [Benchmark Planner Decisions with DynoSim](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/planner-replay.md): Compare topology and scaling decisions against a saved trace
- [AISimulate (Experimental)](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/ai-simulate/overview.md): Backend-neutral simulation and configuration-search tools
- [Sweeper](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/ai-simulate/sweeper/overview.md): Experimental backend-neutral configuration search
- [Sweeper Quickstart](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/ai-simulate/sweeper/quickstart.md): Run a backend-neutral sweep with an injected replay runtime
- [Sweeper Tutorial](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/ai-simulate/sweeper/tutorial.md): Configure, execute, and inspect a replay-backed search
- [Sweeper Architecture](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/ai-simulate/sweeper/architecture.md): Backend-neutral search, provider materialization, and replay execution
- [Sweeper Configuration](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/ai-simulate/sweeper/configuration.md): Core fields and optional adapter-owned search spaces
- [Sweeper Optimization Goals](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/ai-simulate/sweeper/optimization-goals.md): Objective metrics, SLA constraints, and Pareto-front scoring
- [Sweeper Traffic](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/ai-simulate/sweeper/traffic.md): Trace, request-rate, concurrency, and KV-load workload definitions
- [Sweeper Results](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/ai-simulate/sweeper/results.md): Replay specifications, ranked candidates, and Pareto fronts
- [Sweep Configuration Providers](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/ai-simulate/sweeper/sweep-configuration-providers.md): Extend Sweeper without adding application dependencies to its core
- [Dynamo Sweeper Integration](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/ai-simulate/sweeper/dynamo-integration.md): Install and compose Planner, Router, and Dynamo Replay with AISimulate
- [GLM-5-FP8 Disaggregated Pareto Sweep](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/ai-simulate/sweeper/glm-5-fp-8-pareto-sweep.md): Experimental Sweeper search for a GLM-5-FP8 Pareto frontier on B200 GPUs
- [Planner Goodput-per-GPU Sweep](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/ai-simulate/sweeper/planner-goodput-per-gpu-sweep.md): Experimental Sweeper search over Planner policies for MiniMax-M2.5 on B200 GPUs
- [Router End-to-End Latency Sweep](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/simulation/ai-simulate/sweeper/router-end-to-end-latency-sweep.md): Experimental Sweeper search over KV router settings for MiniMax-M2.5 on B200 GPUs
- [Request Migration Architecture](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/fault-tolerance/request-migration-architecture.md): Preserve token state and reissue in-flight requests on healthy workers after a worker failure.
- [Request Cancellation Architecture](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/fault-tolerance/request-cancellation-architecture.md): Cancel in-flight work across the Frontend and workers when a client disconnects.
- [Request Rejection Architecture](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/fault-tolerance/request-rejection-architecture.md): Worker-load event processing, busy-state aggregation, overload errors, and hard worker admission limits.
- [Graceful Shutdown Architecture](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/fault-tolerance/graceful-shutdown-architecture.md)
- [Fault Tolerance Testing](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/fault-tolerance/fault-tolerance-testing.md): Pytest infrastructure for cancellation, migration, etcd high availability, and hardware fault injection.
- [Sidecar Backends](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/concepts/system-architecture/sidecar-backends.md): Run Dynamo beside a stock inference engine through its native gRPC API.
- [Frontend](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/frontend/overview.md): API gateway that exposes OpenAI HTTP and KServe gRPC endpoints and handles request preprocessing, routing, and response formatting.
- [KServe gRPC Frontend](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/frontend/kserve-grpc-frontend.md): Configure KServe endpoints, backend registration, message conversion, and HTTP/2 flow control
- [Python Route Extensions](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/frontend/python-route-extensions.md): Add trusted HTTP routes to the Dynamo Frontend from an installed Python package or importable module.
- [Tokenizer](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/frontend/tokenizer.md): Selects the HuggingFace, fastokens, or Baseten tokenizer backend for BPE models served through the Dynamo Frontend.
- [Router](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/overview.md): Choose a request path, understand worker selection, and operate KV-aware routing.
- [Router Guide](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/router-guide.md): Choose where routing runs and how worker-set configuration takes effect.
- [Routing Concepts](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/routing-concepts.md): Cost model, worker selection, and routing primitives for the Dynamo router
- [Router Filtering](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/router-filtering.md): Candidate eligibility, DP-rank filtering, and queue backpressure boundaries
- [Deficit Round Robin Queue Scheduling](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/deficit-round-robin.md): Weighted arbitration across router policy classes
- [Configuration and Tuning](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/configuration-and-tuning.md): Router behavior, event transport, load tracking, and tuning guidance
- [Router Operations](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/router-operations.md): Replica topology, state management, recovery, and remote indexers
- [KV Event Replay — Dynamo vs vLLM](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/kv-event-replay-comparison.md): How the two systems handle gap detection, replay, and recovery for KV cache events
- [Global Router Configuration](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/global-router-configuration.md): Field reference for the CLI flags and JSON config file consumed by the Global Router service.
- [Disaggregated Serving](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/disaggregated-serving.md): Prefill and decode routing with the Dynamo router
- [Topology-Aware KV Transfer](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/topology-aware-kv-transfer.md): Runtime metadata and decode routing semantics for topology-aware prefill/decode handoff
- [Offloading Support Matrix](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/offloading-support-matrix.md): Per-framework support and setup for KV routing with tiered KV cache offloading
- [Multi-DC KV Routing and the DC Relay](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/multi-dc-kv-routing.md): Aggregate exact KV ownership in each DC and publish compact state to a global consumer
- [Standalone KV Indexer](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/standalone-indexer.md): Run the KV cache indexer as an independent HTTP service for querying block state
- [Standalone Slot Tracker](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/standalone-slot-tracker.md): Run active-request load accounting as an independent HTTP service
- [Standalone Selection Service](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/standalone-selection.md): Select workers and account for reservations without forwarding inference requests
- [Write Custom Routing Strategies](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/write-custom-routing-strategies.md): Build Rust filters, scorers, and pickers for the Dynamo frontend or EPP
- [Router Examples](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/router-examples.md): Worked examples for the KvRouter Python API, Kubernetes deployments, and custom routing patterns.
- [Router Testing](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/router-testing.md): Test layers for router changes
- [Router Design](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/router/router-design.md)
- [Planner](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/planner/overview.md): Autoscaler that adjusts prefill and decode replicas using engine performance models and traffic prediction to meet TTFT and ITL SLAs.
- [Planner Guide](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/planner/planner-guide.md): Configures Planner optimization targets, scaling modes, and PlannerConfig fields for production deployments.
- [Global Planner Deployment Guide](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/planner/global-planner-guide.md): Deploys GlobalPlanner as the centralized scaling layer for multi-DGD policy enforcement and single-endpoint multi-pool deployments.
- [Planner Examples](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/planner/planner-examples.md): Examples for custom load predictors and the VirtualConnector for non-Kubernetes scaling environments.
- [Planner Design](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/planner/planner-design.md)
- [Profiler](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/profiler/overview.md): Automated performance analysis that finds optimal prefill and decode parallelism and generates SLA-driven deployment configurations.
- [Profiler Guide](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/profiler/profiler-guide.md): Runs the AIConfigurator-driven profiling pipeline that turns a model, SLA targets, and backend into a ready-to-deploy DGD.
- [Profiler Examples](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/profiler/profiler-examples.md): Complete DGDR profiling examples for dense and MoE models across rapid and thorough search strategies.
- [SGLang](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/sg-lang/overview.md): SGLang engines run in Dynamo's distributed runtime with disaggregated serving, KV-aware routing, and request cancellation.
- [SGLang Sidecar](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/sg-lang/sidecar.md): Run Dynamo beside a stock SGLang engine through native gRPC.
- [Reference Guide](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/sg-lang/reference-guide.md): Architecture, configuration, and operational details for the SGLang backend
- [SGLang for Agentic Workloads](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/sg-lang/agents-on-sg-lang.md): Priority scheduling and session control for multi-turn agentic serving
- [SGLang Chat Processor](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/sg-lang/chat-processor.md): SGLang-native preprocessing and postprocessing for chat completions
- [Disaggregation](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/sg-lang/disaggregation.md): SGLang runs prefill and decode as separate workers in Dynamo, coordinated by a bootstrap handshake with RDMA-based KV cache transfer.
- [Using HiCache](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/sg-lang/hi-cache.md): Hierarchical KV caching with tier-aware router integration
- [Observability](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/sg-lang/observability.md): Prometheus metrics, forward pass telemetry, distributed tracing, and Grafana dashboards for SGLang workers in Dynamo.
- [SGLang Multimodal](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/sg-lang/sglang-multimodal.md)
- [TensorRT-LLM](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/tensor-rt-llm/overview.md): TensorRT-LLM engines run in Dynamo's distributed runtime with disaggregated serving, KV-aware routing, and multinode support.
- [TensorRT-LLM Sidecar](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/tensor-rt-llm/sidecar.md): Run Dynamo beside a stock TensorRT-LLM engine through native gRPC.
- [Reference Guide](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/tensor-rt-llm/reference-guide.md): Features, configuration, and operational details for the TensorRT-LLM backend
- [TensorRT-LLM Observability](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/tensor-rt-llm/observability.md): TensorRT-LLM exposes request, latency, and KV cache transfer metrics through Dynamo's Prometheus /metrics endpoint.
- [Known Issues and Mitigations](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/tensor-rt-llm/known-issues.md): Known issues for TensorRT-LLM in Dynamo, including disaggregated KV cache exhaustion and driver mismatch errors, with mitigations.
- [TensorRT-LLM Multimodal](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/tensor-rt-llm/tensorrt-llm-multimodal.md)
- [Triton](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/triton/overview.md)
- [vLLM](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/v-llm/overview.md)
- [vLLM Sidecar](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/v-llm/sidecar.md): Run Dynamo beside a stock vLLM engine through native gRPC.
- [Reference Guide](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/v-llm/reference-guide.md): Configuration, arguments, and operational details for the vLLM backend
- [vLLM Chat Processor](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/v-llm/chat-processor.md): vLLM-native preprocessing and postprocessing for chat completions
- [KV Cache Offloading](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/v-llm/kv-cache-offloading.md): CPU and disk offloading integrations for vLLM in Dynamo
- [Native KV Offloading](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/v-llm/native-kv-offloading.md): vLLM's OffloadingConnector with KV-aware routing in Dynamo
- [vLLM Observability](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/v-llm/observability.md): vLLM exposes request, latency, scheduler, and disaggregation metrics through Dynamo's Prometheus /metrics endpoint.
- [vLLM Multimodal](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/v-llm/vllm-multimodal.md)
- [vLLM-Omni Disaggregated Multi-Stage Serving](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/v-llm/v-llm-omni-disaggregated-serving.md): Architecture of multi-stage vLLM-Omni pipelines (e.g. AR + Diffusion), with each stage on its own GPU for independent scaling and isolation.
- [Mocker Engine Architecture](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/knowledge-base/modular-components/backends/mocker/mocker-engine-architecture.md)
- [Conditional Disaggregation](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/advanced-customizations/conditional-disaggregation.md): Route selected requests to local prefill and decode on decode workers
- [Building from Source](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/advanced-customizations/building-from-source.md): Build Dynamo from source for development and contributions
- [Custom Backend Overview](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/advanced-customizations/writing-custom-backends/overview.md): Choose the right path for bringing your own engine to Dynamo
- [Writing Unified Backends](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/advanced-customizations/writing-custom-backends/writing-unified-backends.md): Choose Python or Rust for Dynamo's shared backend contract
- [Writing Python Workers in Dynamo](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/advanced-customizations/writing-custom-backends/writing-python-workers.md): Create custom Python workers and engines for Dynamo
- [Publish KV Events from a Custom Backend](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/advanced-customizations/writing-custom-backends/publish-kv-events.md): Report cache allocation and eviction events to Dynamo's KV router
- [Runtime Containers](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/advanced-customizations/writing-custom-backends/runtime-containers.md): Build Dynamo runtime images for built-in or custom backends
- [Metrics Developer Guide](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/advanced-customizations/metrics-developer-guide.md): Reference for creating counters, gauges, and histograms with Dynamo's metrics API, surfaced on the local /metrics endpoint.
- [Developing the Operator with Tilt](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/advanced-customizations/developing-with-tilt.md): Fast, live-reload development loop for the Dynamo Kubernetes operator
- [Reinforcement Learning](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reinforcement-learning/overview.md): Use Dynamo as the rollout-serving layer for RL workloads
- [KV-Aware Load Balancing for RL Rollouts](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reinforcement-learning/kv-aware-load-balancing.md): Balance cache reuse, live worker load, and queue pressure
- [Distribute and Update Rollout Weights](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reinforcement-learning/rollout-weight-updates.md): Move policy weights into the fleet, then coordinate refresh and recovery
- [Profile and Simulate RL Rollouts](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reinforcement-learning/rl-profiling-and-simulation.md): Inspect a live rollout in Perfetto, then replay its request-plane workload
- [verl Integration](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reinforcement-learning/verl-integration.md): Run the public verl-recipe Dynamo rollout backend
- [NeMo RL Integration](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reinforcement-learning/ne-mo-rl-integration.md): Run NeMo RL's managed Dynamo vLLM backend on Slurm
- [Slime Integration](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reinforcement-learning/slime-integration.md): Run Slime external rollouts through Dynamo on Kubernetes
- [External Trainer Integration](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reinforcement-learning/external-trainer-integration.md): Connect a trainer to Dynamo generation, worker control, and policy updates
- [RL Integration Reference](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reinforcement-learning/rl-integration-reference.md): Generation, worker discovery, administration, and compatibility contracts for RL frameworks
- [Agents](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/agents/overview.md): Agent-aware serving features in Dynamo
- [Session IDs](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/agents/session-i-ds.md): Identify agent sessions from supported coding agents and custom clients
- [Agent Hints](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/agents/agent-hints.md): Per-request serving hints for agentic workloads
- [Priority Scheduling](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/agents/priority-scheduling.md): Request priority across the Dynamo router and backend engines
- [Agent Harnesses](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/agents/agent-harnesses.md): Point coding-agent CLIs at a Dynamo deployment
- [Agent Tracing](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/agents/agent-tracing.md): Export Dynamo request traces, tool-call metadata, and Perfetto timelines
- [Agent Trace Replay](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/agents/agent-simulation.md): Capture an agent workload once, then replay it on real or simulated infrastructure
- [ThunderAgent Program Scheduler](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/agents/thunder-agent-program-scheduler.md): Program-level scheduling with tool-boundary pause/resume on top of KV-aware routing
- [Multimodal Model Serving](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/multimodal/overview.md): Deploy multimodal models with image, video, and audio support in Dynamo
- [Parallel Media Decoding](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/multimodal/parallel-media-decoding.md): Decode image inputs concurrently in the Rust frontend and transfer pixels to inference backends
- [Custom Vision Encoders](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/multimodal/custom-vision-encoders.md): Run a bespoke vision tower in an aggregated Dynamo vLLM worker
- [Embedding Cache](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/multimodal/embedding-cache.md): Cache vision encoder embeddings to skip re-encoding repeated multimodal content
- [Multimodal KV Routing](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/multimodal/multimodal-kv-routing.md): Include multimodal identity in the router's combined cache-and-load cost
- [Encoder Disaggregation](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/multimodal/encoder-disaggregation.md): Separate vision encoding into a dedicated worker for independent scaling
- [Video Decode GPU Requirements](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/multimodal/video-decode-gpu-requirements.md)
- [Additional Media Decoders](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/multimodal/additional-media-decoders.md): Explicitly install software decoders for input formats the image cannot decode
- [Diffusion](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/diffusion/overview.md): Deploy diffusion models for image, video, audio, and text generation
- [Text-to-Image](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/diffusion/text-to-image.md): Generate images from text prompts with vLLM-Omni, SGLang, or TensorRT-LLM
- [Text-to-Video](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/diffusion/text-to-video.md): Generate videos from text prompts with vLLM-Omni, SGLang, TensorRT-LLM, or FastVideo
- [Image-to-Video](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/diffusion/image-to-video.md): Animate source images with vLLM-Omni or SGLang
- [Text-to-Audio](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/diffusion/text-to-audio.md): Synthesize speech with vLLM-Omni through the /v1/audio/speech endpoint
- [Text-to-Text (LLM Diffusion)](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/diffusion/text-to-text.md): Generate text through iterative refinement with SGLang diffusion language models
- [Overview](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/parsing/overview.md): How Dynamo turns raw model output into OpenAI-compatible tool_calls and reasoning_content
- [Chat Processors](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/parsing/chat-processors.md): Pick who parses model output — the dynamo default, or vLLM/SGLang engine fallback
- [Tool Call Parsing (Dynamo)](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/parsing/tool-call-parsing.md): Connect Dynamo to external tools and services using Dynamo's built-in tool call parsers
- [Structural Tag (Guided Decoding for Tool Calls)](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/parsing/structural-tag.md): Constrain model output to valid tool call format using xgrammar structural tags
- [Reasoning Parsing (Dynamo)](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/parsing/reasoning-parsing.md): Configure Dynamo's built-in reasoning parsers for models that emit thinking content
- [Troubleshooting Tool Calls](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/parsing/troubleshooting-tool-calls.md): Capture raw model output with logprobs so issues can be localized
- [Recipes](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/browse.md): Pick a model, choose the hardware/runtime target, and deploy the matching Dynamo recipe.
- [GLM-5.3-Flash](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/glm-5-3-flash.md): Serve GLM-5.3-Flash with Dynamo and vLLM on GB200 or H200, aggregated or with prefill/decode disaggregation.
- [Kimi-K3](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/kimi-k3.md): Serve Kimi-K3 with Dynamo and vLLM on GB200 or GB300, aggregated or disaggregated over NVLink.
- [Qwen3.8-2.4T-A95B](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/qwen-3-8-2-4t-a95b-fp8.md): Serve Qwen3.8-2.4T-A95B with Dynamo and vLLM or SGLang on GB300 or GB200, aggregated or disaggregated.
- [GLM-5.3/5.2](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/glm-5-2.md): Serve GLM-5.3/5.2 with Dynamo and SGLang on B200 or H200 for long-context agentic workloads.
- [Inkling NVFP4](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/inkling.md): Day-0 SGLang recipe for Thinking Machines' Inkling on 8x B200.
- [Kimi-K2.6](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/kimi-k2-6.md): Serve moonshotai/Kimi-K2.6 with Dynamo and vLLM, tuned per GPU and workload.
- [Nemotron 3.5 Lightning](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/nemotron-3-5-lightning.md): Serve NVIDIA-Nemotron-3.5-Lightning-30B-A3B in NVFP4 or BF16 with Dynamo, vLLM, and experimental TensorRT-LLM targets.
- [Nemotron-3-Ultra](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/nemotron-3-ultra.md): Serve Nemotron-3-Ultra with optimized Dynamo 1.4.0 and vLLM profiles on B200, GB200, and H200.
- [Nemotron-3-Super](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/nemotron-3-super.md): Serve NVIDIA-Nemotron-3-Super-120B-A12B with Dynamo and vLLM, tuned per GPU and workload.
- [GLM-5 NVFP4](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/glm-5-nvfp4.md): Serve GLM-5 NVFP4 with Dynamo and SGLang, disaggregated across five GB200 nodes.
- [DeepSeek-V4-Pro](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/deepseek-v4-pro.md): Serve DeepSeek-V4-Pro with Dynamo and vLLM — aggregated or disaggregated, on B200 or H200, tuned for the agentic workload.
- [DeepSeek-V4-Flash](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/deepseek-v4-flash.md): Serve DeepSeek-V4-Flash with Dynamo and vLLM — aggregated or disaggregated, on B200 or H200, tuned for the agentic workload.
- [GPT-OSS-120B](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/gpt-oss-120b.md): Serve openai/gpt-oss-120b with Dynamo on vLLM (B200/H200) or TensorRT-LLM (GB200), aggregated or disaggregated.
- [Qwen3-32B](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/qwen3-32b.md): Serve Qwen3-32B with Dynamo and vLLM — disaggregated serving with KV-aware routing for multi-turn conversational traffic.
- [Qwen3.5-122B-A10B](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/qwen3-5-122b.md): Serve Qwen3.5-122B-A10B with Dynamo and vLLM on B200 or H200 for long-context agentic workloads.
- [Qwen3-235B-A22B FP8](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/qwen3-235b-a22b-fp8.md): Serve Qwen/Qwen3-235B-A22B-FP8 with Dynamo and TensorRT-LLM on 16 Hopper or Blackwell GPUs.
- [Qwen3-32B FP8](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/qwen3-32b-fp8.md): Serve Qwen/Qwen3-32B-FP8 with Dynamo on TensorRT-LLM or vLLM, aggregated or disaggregated.
- [vLLM Deployment Templates](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/kubernetes-templates/dgd/v-llm.md): Ready-to-apply DynamoGraphDeployment manifests for serving vLLM with Dynamo on Kubernetes.
- [TensorRT-LLM Deployment Templates](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/kubernetes-templates/dgd/tensor-rt-llm.md): Ready-to-apply DynamoGraphDeployment manifests for serving TensorRT-LLM with Dynamo on Kubernetes.
- [SGLang Deployment Templates](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/kubernetes-templates/dgd/sg-lang.md): Ready-to-apply DynamoGraphDeployment manifests for serving SGLang with Dynamo on Kubernetes.
- [DGDR Templates](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/kubernetes-templates/dgdr.md): Ready-to-apply DynamoGraphDeploymentRequest manifests for profiling and generating DynamoGraphDeployments.
- [vLLM Local Deployment Examples](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/cli-templates/v-llm.md): Copyable commands for running the vLLM launch scripts in the Dynamo repository.
- [TensorRT-LLM Local Deployment Examples](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/cli-templates/tensor-rt-llm.md): Copyable commands for running the TensorRT-LLM launch scripts in the Dynamo repository.
- [SGLang Local Deployment Examples](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/cli-templates/sg-lang.md): Copyable commands for running the SGLang launch scripts in the Dynamo repository.
- [Feature Benchmarks](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/recipes/benchmarks.md): Start with the feature or performance claim you want to inspect.
- [Agent Skills](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/agent-skills/overview.md): Repository-native skills that AI coding agents use to develop, deploy, operate, and optimize Dynamo.
- [Deployment and Operations Skills](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/agent-skills/deployment.md): Skills for standing up Dynamo deployments and diagnosing them when they misbehave.
- [Performance Optimization Skills](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/agent-skills/optimization.md): The evidence-driven loop that benchmarks a confirmed baseline and challenges it with candidates.
- [Development and Documentation Skills](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/agent-skills/development.md): Skills for working inside the Dynamo repository: investigation, proposals, review, and docs.
- [Compatibility](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/compatibility.md): Hardware, platform, feature, and mixed-version support for Dynamo
- [Releases](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/releases.md): Release history and per-release notes for every Dynamo release, mirrored from GitHub
- [Dynamo v1.5.0](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/releases/v1-5-0.md): Release notes for Dynamo v1.5.0 (GA Sep 18, 2026), including patch release v1.5.1
- [Dynamo v1.4.0](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/releases/v1-4-0.md): Release notes for Dynamo v1.4.0 (GA Aug 14, 2026), including patch releases v1.4.1 and v1.4.2
- [Dynamo v1.3.0](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/releases/v1-3-0.md): Release notes for Dynamo v1.3.0 (GA Jul 20, 2026), including patch release v1.3.1
- [Dynamo v1.2.0](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/releases/v1-2-0.md): Release notes for Dynamo v1.2.0 (GA Jun 2, 2026), including patch release v1.2.1
- [Dynamo v1.1.0](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/releases/v1-1-0.md): Release notes for Dynamo v1.1.0 (GA May 1, 2026), including patch release v1.1.1
- [Dynamo v1.0.0](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/releases/v1-0-0.md): Release notes for Dynamo v1.0.0 (GA Mar 12, 2026), including patch releases v1.0.1, v1.0.2
- [Known Issues](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/releases/known-issues.md): Known issues per Dynamo release, mirrored from the GitHub release notes
- [Deprecations](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/releases/deprecations.md): Breaking changes, deprecations, and removals across Dynamo releases
- [Release Artifacts](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/release-artifacts.md): Container images, Python wheels, Helm charts, and Rust crates for the current release, plus early access builds and release history.
- [Nightly Releases](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/nightly-releases.md): Nightly container images, Python wheels, install patterns, and current backend versions.
- [Model Early Access Builds](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/model-early-access-builds.md): Per-model early access container builds shipped ahead of stable releases
- [Examples](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/examples.md): Reference deployments for SGLang, TensorRT-LLM, vLLM, and custom Dynamo backends.
- [Glossary](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/glossary.md): Definitions of core Dynamo terms covering disaggregated serving, KV cache, components, and the distributed runtime.
- [API Reference](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/api.md): Language-by-language public surface for Dynamo, discovered from the source tree
- [Python API](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/api/python.md): Public Python surfaces on the Dynamo distributed runtime, statically discovered from the source tree.
- [dynamo._core](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/api/python/_core.md): Rust-backed distributed runtime, KV router, and endpoint bindings.
- [dynamo.runtime](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/api/python/runtime.md): Decorators and re-exports for defining Dynamo workers and endpoints.
- [dynamo.llm](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/api/python/llm.md): High-level LLM primitives for building request pipelines.
- [dynamo.frontend](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/api/python/frontend.md): OpenAI-compatible HTTP frontend, argument parsing, and pre/post-processing.
- [dynamo.common](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/api/python/common.md): Shared configuration groups, storage adapters, and utility helpers.
- [dynamo.health_check](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/api/python/health_check.md): Health-check payload types and environment-driven configuration.
- [dynamo.logits_processing](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/api/python/logits_processing.md): Custom logits processors for LLM token generation.
- [dynamo.planner](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/api/python/planner.md): Scaling connectors, decision types, and Planner configuration.
- [dynamo.router](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/api/python/router.md): Request-router configuration and command-line argument groups.
- [dynamo.mocker](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/api/python/mocker.md): Mock engine used to run Dynamo workflows without a real GPU backend.
- [dynamo.nixl_connect](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/api/python/nixl_connect.md): NIXL RDMA connector primitives for KV cache and tensor transport.
- [Rust API](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/api/rust.md): Published Dynamo crates with release-pinned docs.rs links.
- [DynamoGraphDeployment (DGD) Reference](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/api/kubernetes/dynamo-graph-deployment.md): Field reference for the DynamoGraphDeployment custom resource — the canonical, live description of a Dynamo inference graph.
- [DynamoGraphDeploymentRequest (DGDR) Reference](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/api/kubernetes/dynamo-graph-deployment-request.md): Field reference and lifecycle for the DynamoGraphDeploymentRequest custom resource — Dynamo's deploy-by-intent generator.
- [DynamoComponentDeployment (DCD) Reference](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/api/kubernetes/dynamo-component-deployment.md): Field reference for the DynamoComponentDeployment custom resource — the single Dynamo component that a DynamoGraphDeployment reconciles into pods.
- [API Reference](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/api/kubernetes/full-api-reference.md): Full Kubernetes CRD + operator-config API reference, generated from the operator Go types.
- [Runtime Configuration (DynamoRuntimeConfig)](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/components/runtime-configuration.md): Field reference for the cross-cutting Dynamo runtime CLI flags and environment variables shared by every backend and the frontend.
- [TLS](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/components/tls-configuration.md): Encrypt the TCP request/response streams and NATS connections
- [Frontend Configuration Reference](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/components/frontend-configuration.md): Field reference for the Dynamo Frontend CLI arguments, environment variables, and HTTP endpoints.
- [Planner Configuration (PlannerConfig)](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/components/planner-configuration.md): Field reference for the PlannerConfig JSON/YAML object consumed by the Dynamo Planner service.
- [Profiler Configuration Reference](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/components/profiler-configuration.md): Field reference for the Dynamo profiler CLI arguments — the SLA profiler entry point and the low-level per-endpoint profiler.
- [KVBM Configuration Reference](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/components/kvbm-configuration.md): Field reference for the KV Block Manager environment variables and the layered KVBM_* / TOML configuration system.
- [Mocker CLI Reference](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/components/mocker-cli-reference.md): Command-line flags for live Mocker workers
- [DynoSim Replay CLI Reference](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/components/dyno-sim-replay-cli-reference.md): Command options, YAML sections, constraints, and output for aisimulate predict
- [DynoSim Sweep Reference](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/components/dyno-sim-sweep-reference.md): Configuration domains, optimizer controls, and output for aisimulate recommend
- [Global Router Configuration](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/components/global-router-configuration.md): Field reference for the CLI flags and JSON config file consumed by the Global Router service.
- [Gateway API Routing Reference](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/components/gateway-api-routing.md): Dynamo Endpoint Picker Plugin fields, generated resources, runtime settings, and request contracts.
- [ThunderAgent Scheduler Reference](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/components/thunder-agent-scheduler.md): Control-loop flags, constraints, capacity behavior, and scheduler log records
- [vLLM Configuration (DynamoVllmConfig)](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/backends/v-llm-configuration.md): Field reference for the Dynamo-specific CLI flags and environment variables of the vLLM backend wrapper.
- [vLLM-Omni Configuration](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/backends/v-llm-omni-configuration.md): CLI flag reference for the vLLM-Omni multimodal backend (python -m dynamo.vllm.omni).
- [SGLang Configuration (DynamoSGLangConfig)](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/backends/sg-lang-configuration.md): Field reference for the Dynamo-specific CLI flags and environment variables of the SGLang backend wrapper.
- [TensorRT-LLM Configuration (DynamoTrtllmConfig)](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/backends/tensor-rt-llm-configuration.md): Field reference for the Dynamo-specific CLI flags and environment variables of the TensorRT-LLM backend wrapper.
- [Local Observability Stack Reference](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/observability/local-stack.md): Services, ports, signal flow, and configuration files for the Docker Compose stack
- [Health Check Reference](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/observability/health-checks.md): Health endpoints, status codes, paths, and active-check configuration
- [Logging Reference](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/observability/logging.md): Log formats, levels, correlation fields, and backend controls
- [Forward Pass Metrics Trace Reference](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/observability/forward-pass-metrics-traces.md): Configuration, topology support, file rotation, and the dynamo.fpm.trace.v1 schema
- [Request Trace Reference](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/observability/request-traces.md): Record selection, sinks, rotation, OTLP export, and the dynamo.request.trace.v1 schema
- [Environment Variables](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/observability/environment-variables.md): Every environment variable that controls Dynamo metrics, tracing, logging, health checks, and request capture.
- [Metrics Catalog](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/observability/metrics-catalog.md): Every dynamo_* metric emitted by the frontend, backend workers, and router — name, type, and meaning.
- [Engine Metrics Comparison](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/observability/metrics-comparison.md)
- [Metric Labels](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/observability/metric-labels.md): Every label dimension attached to Dynamo metric series — where each one comes from and the values it can take.
- [Operator Metrics](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/observability/operator-metrics.md): Kubernetes-only dynamo_operator_* metrics and the annotations and Helm values that enable metrics collection.
- [Local Resource Monitor (Local)](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/observability/local-resource-monitor-local.md): Host-side per-process VRAM / PCIe / CPU exporter for engine-startup profiling — metrics and CLI flags.
- [NIXL Connect API](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/nixl-connect/overview.md): Python library for dynamic, RDMA-accelerated data transfers between workers in a Dynamo graph.
- [Overview](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/enterprise/overview.md): Commercial support for a curated set of NVIDIA Dynamo release artifacts.
- [Supported Artifacts](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/enterprise/supported-artifacts.md): The exact container images and Helm chart eligible for NVIDIA Enterprise Support.
- [Install](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/enterprise/install.md): Pull the enterprise images and Helm chart, and reference them from your deployments.
- [Support Coverage](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/enterprise/support-coverage.md): What NVIDIA Enterprise Support commits to for the covered Dynamo artifacts.
- [Scope Boundaries](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/reference/enterprise/scope-boundaries.md): What sits outside the Dynamo Enterprise support scope, and how it relates to what is inside.
- [External publications](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/external-publications.md): Articles, deep dives, and deployment write-ups about NVIDIA Dynamo published by customers and partners.
- [Research publications](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/research-publications.md): Papers that use, extend, or benchmark NVIDIA Dynamo, from the teams building on it and the wider systems community.
- [Dynamo Blog](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/digest.md): Engineering deep dives, architecture notes, and project updates from the NVIDIA Dynamo team.
- [Dynamo Agent Optimization Skills](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/digest/agent-optimization-skills.md): Dynamo now ships agent skills that guide coding agents through deployment optimization: clear objective functions, experimental discipline, and inference domain knowledge.
- [DynoSim: Simulating the Pareto Frontier](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/digest/dynosim-pareto-frontier.md): DynoSim is a workload-driven discrete-event simulation of the NVIDIA Dynamo serving stack for mapping Pareto frontiers before real-cluster validation.
- [NVIDIA Dynamo Snapshot: Fast Startup for Inference Workloads on Kubernetes](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/digest/dynamo-snapshot-fast-startup.md): NVIDIA Dynamo Snapshot combines CUDA and host checkpointing to restore warm inference workers quickly on Kubernetes.
- [Dynamo Day 0 support for TokenSpeed](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/digest/tokenspeed-day-0.md): Dynamo adds day-0 TokenSpeed support with the Dynamo frontend for Kimi K2.5.
- [Streaming Tokens and Tools: Multi-Turn Agentic Harness Support in Dynamo](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/digest/agentic-harnesses.md): Streaming Tokens and Tools: Multi-Turn Agentic Harness Support in Dynamo
- [Full-Stack Optimizations for Agentic Inference with Dynamo](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/digest/agentic-inference.md): How Dynamo optimizes for agentic workloads at three layers: frontend API, router, and KV cache management.
- [Flash Indexer: A Story of Inter-Galactic KV Routing](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/digest/flash-indexer.md): Dynamo's Flash Indexer tracks every cached KV block across all inference workers at 170M ops/s. Six iterations of data structure design got it there.
- [Community](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/community.md): Community meetings, events, and project channels
- [Contributing to Dynamo](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/contributing/contribution-guide.md): Choose a contribution path and find the guide for your next step
- [Contributor Getting Started](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/contributing/getting-started.md): Set up a fork and create a signed, validated change
- [Contribution Flow](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/contributing/contribution-flow.md): Take a change from proposal through pull request review
- [Code Quality](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/contributing/code-quality.md): Style, testing, validation, and commit conventions for Dynamo changes
- [DCO and Licensing](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/contributing/dco-and-licensing.md): Sign, verify, and repair commits for contribution acceptance
- [Documentation Style Guide](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/contributing/documentation/style-guide.md): How to write and structure Dynamo docs, examples, and recipes
- [How to Build and Publish Dynamo Docs](https://docs.dynamo.nvidia.com/dynamo/v1.5.1/contributing/documentation/building-and-publishing.md): Branch model, sync workflow, and publishing pipeline for the Fern-powered Dynamo documentation site.


> **Note:** This page contains both a page directory (above) and the landing page content (below). The page directory is generated for agent use and does not appear on the landing page.

> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Dynamo

![](/dynamo/_fern-files/dynamo.docs.buildwithfern.com/4019188fbddfe63ae7a9ac420a19191e951921157d63b773d2ae3b61ff03728e/assets/img/dynamo-logo.svg)