> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Feature Guides

Use these guides after you have Dynamo running and want to improve serving behavior, operate a deployment, or adapt Dynamo to a new workload.

## Recommended path

Most deployments start with the core performance loop:

| Step | Guide | Use when |
|---|---|---|
| 1 | [KV Cache Aware Routing](/dynamo/v1.3.0/user-guides/kv-cache-aware-routing) | Route requests to workers that already hold useful KV cache. |
| 2 | [Disaggregated Serving](/dynamo/v1.3.0/user-guides/disaggregated-serving) | Scale prefill and decode workers independently. |
| 3 | [KV Cache Offloading](/dynamo/v1.3.0/user-guides/kv-cache-offloading) | Extend usable cache capacity beyond GPU memory. |
| 4 | [Benchmarking](/dynamo/v1.3.0/user-guides/benchmarking) | Compare configurations before you move to production. |

## Where to go next

| Goal | Start with |
|---|---|
| Make serving more resilient | [Fault Tolerance](/dynamo/v1.3.0/user-guides/fault-tolerance) |
| Monitor local deployments | [Observability (Local)](/dynamo/v1.3.0/user-guides/observability-local) |
| Reproduce traffic without a full engine | [Mocker Engine Simulation](../mocker/mocker.md) |
| Add structured model outputs | [Tool Calling](/dynamo/v1.3.0/user-guides/parsing/tool-call-parsing-dynamo) and [Reasoning](/dynamo/v1.3.0/user-guides/parsing/reasoning-parsing-dynamo) |
| Build agent workloads | [Agents](/dynamo/v1.3.0/user-guides/agents) |
| Serve specialized workloads | [LoRA Adapters](/dynamo/v1.3.0/user-guides/lo-ra-adapters), [Multimodal](/dynamo/v1.3.0/user-guides/multimodal), and [Diffusion](/dynamo/v1.3.0/user-guides/diffusion) |

For cluster deployments, pair these guides with the [Kubernetes Deployment](/dynamo/v1.3.0/kubernetes-deployment/start-here/kubernetes-quickstart) docs. The same features can be explored locally, then expressed through Dynamo's Kubernetes-native CRDs and operator when you move to a shared GPU cluster.

> Start with Dynamo's core serving optimizations, then branch into operations and model capabilities.

## Docs

- [Router Guide](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/kv-cache-aware-routing.md): Deployment modes, quick start, and page map for Dynamo routing docs
- [Disaggregated Serving](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/disaggregated-serving.md): Split prefill and decode into independently scalable worker pools
- [Sizing with AIConfigurator](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/disaggregated-serving/sizing-with-ai-configurator.md): Compare aggregated and disaggregated layouts before deployment
- [KVBM Guide](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/kv-cache-offloading.md): Enable KV offloading using KV Block Manager (KVBM) for Dynamo deployments
- [Dynamo Benchmarking](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/benchmarking.md): Benchmark and compare performance across Dynamo deployment configurations
- [Tool Calling & Reasoning Parsing](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/parsing.md): Parse tool calls and reasoning out of model output into OpenAI-compatible tool_calls and reasoning_content
- [Tool Call Parsing (Dynamo)](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/parsing/tool-call-parsing-dynamo.md): Connect Dynamo to external tools and services using Dynamo's built-in tool call parsers
- [Reasoning Parsing (Dynamo)](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/parsing/reasoning-parsing-dynamo.md): Configure Dynamo's built-in reasoning parsers for models that emit thinking content
- [Parser Engine Fallback](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/parsing/parser-engine-fallback.md): Use upstream vLLM or SGLang tool-call and reasoning parsers when Dynamo does not ship one
- [Parser Configuration](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/parsing/parser-configuration.md): How --dyn-chat-processor, --dyn-tool-call-parser, and --dyn-reasoning-parser fit together
- [Tool Calling Probe Snapshot for Dynamo 1.2](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/parsing/tool-calling-probe-snapshot-for-dynamo-1-2.md): Static release snapshot of tool-calling probe results across supported model families
- [Troubleshooting Tool Calls](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/parsing/troubleshooting-tool-calls.md): Capture raw model output with logprobs so issues can be localized
- [Fault Tolerance](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/fault-tolerance.md): Handle failures gracefully with request migration, cancellation, and graceful shutdown
- [Request Migration](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/fault-tolerance/request-migration.md): Preserves token state and reissues in-flight requests on healthy workers when their original worker fails.
- [Request Cancellation](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/fault-tolerance/request-cancellation.md): Cancels in-flight requests across frontend and workers when clients disconnect, freeing GPU resources end to end.
- [Request Rejection](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/fault-tolerance/request-rejection.md): Configure independent worker-load thresholds that shed traffic with HTTP 529
- [Graceful Shutdown](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/fault-tolerance/graceful-shutdown.md): Coordinates SIGTERM and SIGINT handling so endpoints drain, in-flight requests finish, and pods restart cleanly.
- [Testing](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/fault-tolerance/testing.md): Pytest framework for validating Dynamo's cancellation, migration, etcd HA, and hardware fault-injection paths under induced failures.
- [Observability (Local)](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/observability-local.md): Monitor Dynamo deployments with metrics, logging, and tracing
- [Prometheus + Grafana Setup](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/observability-local/prometheus-grafana-setup.md): Single-machine setup for collecting Dynamo metrics in Prometheus and visualizing them in Grafana for local development and demos.
- [Metrics](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/observability-local/metrics.md): Reference for the Prometheus metrics Dynamo's runtime, frontend, and workers expose on the local system metrics endpoint.
- [Metrics Developer Guide](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/observability-local/metrics-developer-guide.md): Reference for creating counters, gauges, and histograms with Dynamo's metrics API, surfaced on the local /metrics endpoint.
- [Health Checks](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/observability-local/health-checks.md): HTTP health and liveness endpoints Dynamo components expose locally for wiring startup, liveness, and readiness probes.
- [Tracing](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/observability-local/tracing.md): OpenTelemetry distributed tracing for Dynamo running locally via Docker Compose, exporting spans to Tempo and visualizing them in Grafana.
- [Forward Pass Metrics Tracing](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/observability-local/forward-pass-metrics-tracing.md): Persist backend forward pass metrics to rotating gzip JSONL files
- [Logging](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/observability-local/logging.md): Structured text and JSONL logging for local Dynamo processes, with optional OTLP export to Loki via the OpenTelemetry Collector.
- [DynoSim](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/dynosim.md): Simulate Dynamo deployment choices before spending GPU time
- [Live Simulation with Mocker](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/dynosim/mocker.md): Mocker simulates production engine scheduling, KV cache, and timing so you can exercise Dynamo's frontend, router, and planner without GPUs.
- [DynoSim Runs](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/dynosim/runs.md): Run one trace or synthetic workload through a simulated Dynamo configuration
- [DynoSim Sweeps](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/dynosim/sweeps.md): Search simulated deployment candidates across topology, router, and SLA constraints
- [Planner DynoSim Benchmarking](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/dynosim/planner-benchmarking.md): Drive the planner in the simulation loop against a saved trace to evaluate SLA behavior and scaling decisions
- [Agents](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/agents.md): Agent-aware serving features in Dynamo
- [Session IDs](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/agents/session-i-ds.md): Identify agent sessions from supported coding agents and custom clients
- [Agent Tracing](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/agents/agent-tracing.md): Export Dynamo request traces, tool-call metadata, and Perfetto timelines
- [Agent Trace Replay](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/agents/agent-simulation.md): Capture an agent workload once, then replay it on real or simulated infrastructure
- [Agent Hints](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/agents/agent-hints.md): Per-request serving hints for agentic workloads
- [Agent Harnesses](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/agents/agent-harnesses.md): Point coding-agent CLIs at a Dynamo deployment
- [ThunderAgent Program Scheduler](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/agents/thunder-agent-program-scheduler.md): Program-level scheduling with tool-boundary pause/resume on top of KV-aware routing
- [LoRA Adapters](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/lo-ra-adapters.md): Serve fine-tuned LoRA adapters with dynamic loading and routing in Dynamo
- [Multimodal Model Serving](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/multimodal.md): Deploy multimodal models with image, video, and audio support in Dynamo
- [Embedding Cache](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/multimodal/embedding-cache.md): Cache vision encoder embeddings to skip re-encoding repeated multimodal content
- [Encoder Disaggregation](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/multimodal/encoder-disaggregation.md): Separate vision encoding into a dedicated worker for independent scaling
- [Custom Vision Encoders](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/multimodal/custom-vision-encoders.md): Run a bespoke vision tower in an aggregated Dynamo vLLM worker
- [Multimodal KV Routing](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/multimodal/multimodal-kv-routing.md): Route multimodal requests to workers with the best KV cache overlap
- [SGLang Multimodal](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/multimodal/sg-lang-multimodal.md)
- [TensorRT-LLM Multimodal](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/multimodal/tensor-rt-llm-multimodal.md)
- [vLLM Multimodal](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/multimodal/v-llm-multimodal.md)
- [Diffusion](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/diffusion.md): Deploy diffusion models for text-to-image, text-to-video, and more in Dynamo
- [FastVideo](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/diffusion/fastvideo.md): Deploys FastVideo text-to-video generation on Dynamo through a custom worker that serves the /v1/videos endpoint.
- [Fastokens Tokenizer](https://docs.nvidia.com/dynamo/v1.3.0/user-guides/fastokens-tokenizer.md): Reduce frontend tokenization latency for long-context BPE models