> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt. # Compatibility The card above covers one release at a time. To compare CUDA toolkit and minimum driver requirements across releases per backend, jump to the [Release Support Matrix](#release-support-matrix) at the bottom of this page. For extended driver compatibility beyond the listed minimums, including forward compatibility and `cuda-compat` packages, see the [CUDA Compatibility documentation](https://docs.nvidia.com/deploy/cuda-compatibility/latest/). See [Release Artifacts](/dynamo/reference/release-artifacts) for the full artifact inventory — container images, wheels, Helm charts, and crates — [Local Installation](/dynamo/dev/cli/installation/install-dynamo) for host OS and architecture requirements, and [Model Early Access Builds](/dynamo/reference/model-early-access-builds) for per-model early access container builds. For backend-specific runtime workarounds — including the local `docker run --network host` hang first reported on Amazon Linux 2023 — see [TensorRT-LLM Known Issues](/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/known-issues). ## Mixed-Version Compatibility Dynamo supports mixed-version operation between frontends and workers across an N-2 window: the current release and the two immediately previous release lines. Any frontend and worker combination within that window is supported in both age directions. For example, a current frontend can serve workers from either previous release, and a frontend from either previous release can serve current workers. A single frontend can discover worker generations from multiple supported releases for the same logical deployment. N-3 and older combinations are unsupported unless a narrower exception is explicitly documented. An explicitly enabled feature may also reject a mixed-version combination when its semantics cannot be represented safely by the other version. This guarantee covers the frontend-to-worker discovery metadata and wire protocols owned by Dynamo. It does not establish a compatibility window for direct worker-to-worker protocols, such as prefill-to-decode communication. ### Rolling Update Behavior In Kubernetes deployments, worker spec changes during a rolling update create generation-specific Dynamo runtime namespaces (service-discovery scopes, not Kubernetes namespaces). The frontend watches the deployment's base runtime namespace prefix, so it can discover overlapping generations without allowing direct worker-to-worker communication across those generations. In a disaggregated deployment, prefill and decode workers in the same generation share one runtime namespace and remain isolated from other generations. The frontend excludes incomplete or unready namespaces from routing. When multiple ready WorkerSets are available, it selects a WorkerSet at random with weight proportional to its worker count, then applies the configured routing policy within that WorkerSet. Worker count is a capacity approximation, not a measurement of end-to-end serving capacity. Selection between WorkerSets also does not provide request stickiness. ## Feature Support ### Per-Backend Detail #### vLLM vLLM offers the broadest feature coverage in Dynamo, with full support for disaggregated serving, KV-aware routing, KV block management, LoRA adapters, and multimodal inference including video and audio. *Source: [docs/backends/vllm/README.md][vllm-readme]* | Feature | Supported? | Notes | | :----------------------------------------------------------------------------------------------------------- | :--------: | :---------------------------------------------------------------------------------------------------------------------------------------- | | **Disaggregated Serving** | ✓ | Prefill/decode separation with NIXL KV transfer | | **KV-Aware Routing** | ✓ | | | **SLA-Based Planner** | ✓ | | | **KV Block Manager** | ✓ | | | **Multimodal** | ✓ | Image + video; audio experimental (Qwen2-Audio). With KV-aware routing, image-aware routing on documented paths ([Source][mm-kv-routing]) | | **Request Migration** | ✓ | | | **Request Cancellation** | ✓ | | | **LoRA** | ✓ | Dynamic load/unload; KV-aware routing supports adapter affinity | | **Tool Calling** | ✓ | | | **Speculative Decoding** | ✓ | Eagle3 ([Source][vllm-spec]) | | **GPU Memory Service** | ✓ | Weights and KV; upstream integration remains in progress | | **Shadow Engine Failover** | ! | Software-process failover only; no KV-cache reuse or hardware fault tolerance | | **[Dynamo Snapshot API](/dynamo/dev/reference/api/kubernetes/full-api-reference#componentcheckpointconfig)** | ! | Single-GPU supported; multi-GPU is highly experimental and multinode remains in progress | #### SGLang SGLang is optimized for high-throughput serving with fast primitives, providing robust support for disaggregated serving, KV-aware routing, and request migration. *Source: [docs/backends/sglang/README.md][sglang-readme]* | Feature | Supported? | Notes | | :----------------------------------------------------------------------------------------------------------- | :--------: | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Disaggregated Serving** | ✓ | | | **KV-Aware Routing** | ✓ | | | **SLA-Based Planner** | ✓ | | | **KV Block Manager** | WIP | Work in progress across all combinations | | **Multimodal** | ✓ | Image + video. KV-aware routing supported on Dynamo's SGLang image; a custom build without the hash-forwarding patch falls back to text-prefix routing. Disagg patterns: EPD, E/PD, E/P/D (not traditional EP/D) ([Source][mm-sglang]) | | **Request Migration** | ✓ | | | **Request Cancellation** | WIP | Remote-prefill-phase cancellation not supported in disaggregated mode ([Source][sglang-readme]) | | **LoRA** | — | Not supported | | **Tool Calling** | ✓ | | | **Speculative Decoding** | WIP | Code hooks exist; no examples or docs yet | | **GPU Memory Service** | ✓ | Weights and KV; upstream integration remains in progress | | **Shadow Engine Failover** | WIP | Experimental; no KV-cache reuse or hardware fault tolerance | | **[Dynamo Snapshot API](/dynamo/dev/reference/api/kubernetes/full-api-reference#componentcheckpointconfig)** | ! | Single-GPU supported; multi-GPU and multinode remain in progress | #### TensorRT-LLM TensorRT-LLM delivers maximum inference performance and optimization, with full KVBM integration and robust disaggregated serving support. *Source: [docs/backends/trtllm/README.md][trtllm-readme]* | Feature | Supported? | Notes | | :----------------------------------------------------------------------------------------------------------- | :--------: | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Disaggregated Serving** | ✓ | | | **KV-Aware Routing** | ✓ | | | **SLA-Based Planner** | ✓ | | | **KV Block Manager** | ✓ | | | **Multimodal** | ✓ | Image only (URLs + pre-computed embeddings). Disagg: EP/D + E/P/D. With KV-aware routing, via dedicated MM Router Worker (requires KV event publishing) ([Source][mm-trtllm]) | | **Request Migration** | ✓ | Work in progress with multimodal | | **Request Cancellation** | ! | Engine temporarily not notified of cancellations — resources for cancelled requests are not freed (known issue) | | **LoRA** | — | Not supported | | **Tool Calling** | ✓ | | | **Speculative Decoding** | ✓ | | | **GPU Memory Service** | WIP | Weights only; multinode and upstream integration remain in progress | | **Shadow Engine Failover** | WIP | Experimental; no KV-cache reuse or hardware fault tolerance | | **[Dynamo Snapshot API](/dynamo/dev/reference/api/kubernetes/full-api-reference#componentcheckpointconfig)** | WIP | Experimental single-GPU aggregated text-worker path only | ### Feature Interactions Pairwise feature-by-feature compatibility within each backend. Each cell reports whether the row feature works together with the column feature. #### vLLM #### SGLang #### TensorRT-LLM [vllm-readme]: ../../developer-guide/knowledge-base/modular-components/backends/vllm/overview.md [sglang-readme]: ../../developer-guide/knowledge-base/modular-components/backends/sglang/overview.md [trtllm-readme]: ../../developer-guide/knowledge-base/modular-components/backends/tensorrt-llm/overview.md [mm-kv-routing]: ../../use-cases/multimodal-serving/multimodal-kv-routing.md [mm-sglang]: ../../developer-guide/knowledge-base/modular-components/backends/sglang/multimodal.md [mm-trtllm]: ../../developer-guide/knowledge-base/modular-components/backends/tensorrt-llm/multimodal.md [vllm-spec]: ../../developer-guide/additional-resources/speculative-decoding/speculative-decoding-with-vllm.md ## Release Support Matrix Every stable release and patch, grouped by minor line — newest first. Expand a line to see each release's backend pins, CUDA toolkit, and minimum driver, ordered CUDA 12 before CUDA 13. Releases predating per-release CUDA tracking are listed with their requirements marked "Not recorded" rather than dropped. Platform previews and model-specific builds are excluded; the notes below call out the ones whose toolkit support differs, and the [Releases (machine-readable)](/dynamo/reference/releases-data) page has the full inventory. Driver already installed? Read across from your version — each cell is the newest release that backend can run on it. A driver meeting a higher floor also runs everything below it. | Driver | SGLang | TensorRT-LLM | vLLM | | ------- | ------ | ------------ | ----------- | | 570.xx+ | 0.7.1 | None | 0.7.0.post1 | | 575.xx+ | 1.2.1 | None | 1.2.1 | | 580.xx+ | 1.4.2 | 1.4.2 | 1.4.2 | Current stable release: v1.4.2 (container tag `1.4.2`, wheel version `1.4.2`). **Backend engine pins per Dynamo release** | Dynamo | Type | SGLang | TensorRT-LLM | vLLM | NIXL (SGL / TRT / vLLM) | UCX | | ------------------------ | ---------------- | --------------------- | -------------- | ------ | ------------------------ | ------ | | main (ToT) | development head | 0.5.19 | 1.3.0rc25 | 0.28.0 | 1.4.0 / 1.3.1 / 1.3.2 | - | | v1.4.2 | patch | 0.5.16 | 1.3.0rc22 | 0.26.0 | 1.3.0 / 1.3.1 / 1.3.2 | 1.21.x | | v1.4.1 | patch | 0.5.16 | 1.3.0rc22 | 0.26.0 | 1.3.0 / 1.3.1 / 1.3.2 | 1.21.x | | v1.4.0 | stable | 0.5.16 | 1.3.0rc22 | 0.26.0 | 1.3.0 / 1.3.1 / 1.3.2 | 1.21.x | | v1.3.1 | patch | 0.5.14 | 1.3.0rc19 | 0.23.0 | 1.3.2 / 1.0.1 / 1.1.0 | 1.20.x | | v1.3.0 | stable | 0.5.14 | 1.3.0rc19 | 0.23.0 | 1.3.0 / 1.0.1 / 1.1.0 | 1.20.x | | v1.3.0-dev.1 | platform-preview | 0.5.12.post1 | 1.3.0rc17 | 0.22.0 | 1.0.1 / 0.10.1 / 1.1.0 | - | | v1.2.1 | patch | 0.5.11 | 1.3.0rc14 | 0.20.1 | 1.0.1 / 0.10.1 / 0.10.1 | - | | v1.2.0 | stable | 0.5.11 | 1.3.0rc14 | 0.20.1 | 1.0.1 / 0.10.1 / 0.10.1 | 1.20.0 | | v1.2.0-deepseek-v4-dev.3 | model-build | upstream DSv4 preview | - | 0.20.1 | - / - / 0.10.1 | - | | v1.2.0-deepseek-v4-dev.2 | model-build | upstream DSv4 preview | - | 0.20.0 | - / - / 0.10.1 | - | | v1.1.1 | patch | 0.5.10.post1 | 1.3.0rc11 | 0.19.0 | 1.0.1 / 0.10.1 / 0.10.1 | - | | v1.1.0 | stable | 0.5.10.post1 | 1.3.0rc11 | 0.19.0 | 1.0.1 / 0.10.1 / 0.10.1 | 1.20 | | v1.1.0-dev.3 | platform-preview | 0.5.10.post1 | 1.3.0rc11 | 0.19.0 | 1.0.1 / 0.10.1 / 0.10.1 | - | | v1.1.0-dev.2 | platform-preview | 0.5.9 | 1.3.0rc9 | 0.19.0 | 1.0.1 / 0.10.1 / 0.10.1 | - | | v1.1.0-dev.1 | platform-preview | 0.5.9 | 1.3.0rc5.post1 | 0.17.1 | 1.0.1 / 0.10.1 / 0.10.1 | - | | v1.0.2 | patch | 0.5.9 | 1.3.0rc5.post1 | 0.16.0 | 0.10.1 / 0.10.1 / 0.10.1 | - | | v1.0.1 | patch | 0.5.9 | 1.3.0rc5.post1 | 0.16.0 | 0.10.1 / 0.10.1 / 0.10.1 | - | | v1.0.0 | stable | 0.5.9 | 1.3.0rc5.post1 | 0.16.0 | 0.10.1 / 0.10.1 / 0.10.1 | - | | v0.9.1 | patch | 0.5.8 | 1.3.0rc3 | 0.14.1 | 0.9.0 / 0.9.0 / 0.9.0 | - | | v0.9.0 | stable | 0.5.8 | 1.3.0rc1 | 0.14.1 | 0.9.0 / 0.9.0 / 0.9.0 | - | | v0.8.1.post3 | patch | 0.5.6.post2 | 1.2.0rc6.post3 | 0.12.0 | 0.8.0 / 0.8.0 / 0.8.0 | - | | v0.8.1.post2 | patch | 0.5.6.post2 | 1.2.0rc6.post2 | 0.12.0 | 0.8.0 / 0.8.0 / 0.8.0 | - | | v0.8.1.post1 | patch | 0.5.6.post2 | 1.2.0rc6.post1 | 0.12.0 | 0.8.0 / 0.8.0 / 0.8.0 | - | | v0.8.1 | patch | 0.5.6.post2 | 1.2.0rc6.post1 | 0.12.0 | 0.8.0 / 0.8.0 / 0.8.0 | - | | v0.8.0 | stable | 0.5.6.post2 | 1.2.0rc6.post1 | 0.12.0 | 0.8.0 / 0.8.0 / 0.8.0 | - | | v0.7.1 | patch | 0.5.4.post3 | 1.2.0rc3 | 0.11.0 | 0.8.0 / 0.8.0 / 0.8.0 | - | | v0.7.0.post1 | patch | 0.5.4.post3 | 1.2.0rc3 | 0.11.0 | 0.8.0 / 0.8.0 / 0.8.0 | - | | v0.7.0 | stable | 0.5.4.post3 | 1.2.0rc2 | 0.11.0 | 0.8.0 / 0.8.0 / 0.8.0 | - | | v0.6.1.post1 | patch | 0.5.3.post2 | 1.1.0rc5 | 0.11.0 | 0.6.0 / 0.6.0 / 0.6.0 | - | | v0.6.1 | patch | 0.5.3.post2 | 1.1.0rc5 | 0.11.0 | 0.6.0 / 0.6.0 / 0.6.0 | - | | v0.6.0 | stable | 0.5.3.post2 | 1.1.0rc5 | 0.11.0 | 0.6.0 / 0.6.0 / 0.6.0 | - | **CUDA toolkit and minimum driver per Dynamo release** | Dynamo | Backend | CUDA Toolkit | Min Driver | Note | | ------ | ------------ | ------------ | ---------- | ------------ | | 1.4.2 | SGLang | 13.0 | 580.xx+ | - | | 1.4.2 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.4.2 | vLLM | 13.0 | 580.xx+ | - | | 1.4.1 | SGLang | 13.0 | 580.xx+ | - | | 1.4.1 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.4.1 | vLLM | 13.0 | 580.xx+ | - | | 1.4.0 | SGLang | 13.0 | 580.xx+ | - | | 1.4.0 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.4.0 | vLLM | 13.0 | 580.xx+ | - | | 1.3.1 | SGLang | 13.0 | 580.xx+ | - | | 1.3.1 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.3.1 | vLLM | 13.0 | 580.xx+ | - | | 1.3.0 | SGLang | 13.0 | 580.xx+ | - | | 1.3.0 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.3.0 | vLLM | 13.0 | 580.xx+ | - | | 1.2.1 | SGLang | 12.9 | 575.xx+ | - | | 1.2.1 | SGLang | 13.0 | 580.xx+ | - | | 1.2.1 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.2.1 | vLLM | 12.9 | 575.xx+ | - | | 1.2.1 | vLLM | 13.0 | 580.xx+ | - | | 1.2.0 | SGLang | 12.9 | 575.xx+ | - | | 1.2.0 | SGLang | 13.0 | 580.xx+ | - | | 1.2.0 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.2.0 | vLLM | 12.9 | 575.xx+ | - | | 1.2.0 | vLLM | 13.0 | 580.xx+ | - | | 1.1.1 | SGLang | 12.9 | 575.xx+ | - | | 1.1.1 | SGLang | 13.0 | 580.xx+ | - | | 1.1.1 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.1.1 | vLLM | 12.9 | 575.xx+ | - | | 1.1.1 | vLLM | 13.0 | 580.xx+ | - | | 1.1.0 | SGLang | 12.9 | 575.xx+ | - | | 1.1.0 | SGLang | 13.0 | 580.xx+ | - | | 1.1.0 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.1.0 | vLLM | 12.9 | 575.xx+ | - | | 1.1.0 | vLLM | 13.0 | 580.xx+ | - | | 1.0.2 | SGLang | 12.9 | 575.xx+ | - | | 1.0.2 | SGLang | 13.0 | 580.xx+ | - | | 1.0.2 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.0.2 | vLLM | 12.9 | 575.xx+ | - | | 1.0.2 | vLLM | 13.0 | 580.xx+ | - | | 1.0.1 | SGLang | 12.9 | 575.xx+ | - | | 1.0.1 | SGLang | 13.0 | 580.xx+ | - | | 1.0.1 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.0.1 | vLLM | 12.9 | 575.xx+ | - | | 1.0.1 | vLLM | 13.0 | 580.xx+ | - | | 1.0.0 | SGLang | 12.9 | 575.xx+ | - | | 1.0.0 | SGLang | 13.0 | 580.xx+ | - | | 1.0.0 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.0.0 | vLLM | 12.9 | 575.xx+ | - | | 1.0.0 | vLLM | 13.0 | 580.xx+ | - | | 0.9.1 | SGLang | 12.9 | 575.xx+ | - | | 0.9.1 | TensorRT-LLM | 13.0 | 580.xx+ | - | | 0.9.1 | vLLM | 12.9 | 575.xx+ | - | | 0.9.0 | SGLang | 12.9 | 575.xx+ | - | | 0.9.0 | TensorRT-LLM | 13.0 | 580.xx+ | - | | 0.9.0 | vLLM | 12.9 | 575.xx+ | - | | 0.8.1 | SGLang | 12.9 | 575.xx+ | - | | 0.8.1 | SGLang | 13.0 | 580.xx+ | Experimental | | 0.8.1 | TensorRT-LLM | 13.0 | 580.xx+ | - | | 0.8.1 | vLLM | 12.9 | 575.xx+ | - | | 0.8.1 | vLLM | 13.0 | 580.xx+ | Experimental | | 0.8.0 | SGLang | 12.9 | 575.xx+ | - | | 0.8.0 | SGLang | 13.0 | 580.xx+ | Experimental | | 0.8.0 | TensorRT-LLM | 13.0 | 580.xx+ | - | | 0.8.0 | vLLM | 12.9 | 575.xx+ | - | | 0.8.0 | vLLM | 13.0 | 580.xx+ | Experimental | | 0.7.1 | SGLang | 12.8 | 570.xx+ | - | | 0.7.1 | TensorRT-LLM | 13.0 | 580.xx+ | - | | 0.7.1 | vLLM | 12.9 | 575.xx+ | - | | 0.7.0 | SGLang | 12.9 | 575.xx+ | - | | 0.7.0 | TensorRT-LLM | 13.0 | 580.xx+ | - | | 0.7.0 | vLLM | 12.8 | 570.xx+ | - | * Patch versions (e.g. v0.8.1.post1, v0.7.0.post1) have the same CUDA support as their base version. * Early access v1.1.0-dev.\* images follow the same CUDA matrix as v1.0.2. The v1.2.0-deepseek-v4-dev.3 vLLM container is CUDA 13.0 multi-arch; the SGLang containers split by arch (CUDA 12.9 on amd64, CUDA 13.0 on arm64). * Experimental CUDA 13 images are not published for all versions. **Feature support by backend (v1.4.2)** | Feature | SGLang | TensorRT-LLM | vLLM | | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------- | | Disaggregated Serving | Supported | Supported | Supported (Prefill/decode separation with NIXL KV transfer) | | KV-Aware Routing | Supported | Supported | Supported | | SLA-Based Planner | Supported | Supported | Supported | | KV Block Manager | Experimental (Work in progress across all combinations) | Supported | Supported | | Multimodal (Image) | Supported (KV-aware routing supported on Dynamo's SGLang image for aggregated workers; a custom build without the hash-forwarding patch falls back to text-prefix routing. Separately, multimodal serving supports EPD, E/PD and E/P/D disaggregation (not traditional EP/D)) | Supported (Image URLs + pre-computed embeddings. Disagg: EP/D + E/P/D. KV-aware routing via dedicated MM Router Worker (requires KV event publishing)) | Supported (With KV-aware routing, image-aware routing on documented paths) | | Multimodal (Video) | Supported | Not supported | Supported (Video input with frame sampling) | | Multimodal (Audio) | Not supported | Not supported | Experimental (Qwen2-Audio, experimental) | | Request Migration | Supported | Supported (Work in progress with multimodal) | Supported | | Request Cancellation | Experimental (Remote-prefill-phase cancellation not supported in disaggregated mode) | Supported with caveat (Engine temporarily not notified of cancellations — resources for cancelled requests are not freed (known issue)) | Supported | | LoRA | Experimental (Dynamic loading, discovery, and aggregated inference validated; unloading is implemented but not end-to-end tested; disaggregated serving not end-to-end validated) | Not supported | Supported (Dynamic load/unload; KV-aware routing supports adapter affinity) | | Tool Calling | Supported | Supported | Supported | | Speculative Decoding | Experimental (Code hooks exist; no examples or docs yet) | Supported | Supported (Eagle3) | | GPU Memory Service | Supported (Weights and KV; upstream integration remains in progress) | Experimental (Weights only; multinode and upstream integration remain in progress) | Supported (Weights and KV; upstream integration remains in progress) | | Shadow Engine Failover | Experimental (No KV-cache reuse or hardware fault tolerance) | Experimental (No KV-cache reuse or hardware fault tolerance) | Supported with caveat (Software-process failover only; no KV-cache reuse or hardware fault tolerance) | | Dynamo Snapshot | Supported with caveat (Single-GPU supported; multi-GPU and multinode remain in progress) | Experimental (Single-GPU aggregated text-worker path only) | Supported with caveat (Single-GPU supported; multi-GPU is highly experimental and multinode remains in progress) | **Feature interactions by backend** Each cell states whether the row feature works together with the column feature. The matrix is symmetric: a cell reads the same in either direction. *vLLM* | Feature | Disaggregated Serving | KV-Aware Routing | SLA-Based Planner | KV Block Manager | Multimodal | Request Migration | Request Cancellation | LoRA | Tool Calling | Speculative Decoding | | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- | -------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Disaggregated Serving | n/a | Yes | Yes | Yes | Yes — Supports Qwen2-Audio experimentally and video input with frame sampling. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/v-llm/vllm-multimodal](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/v-llm/vllm-multimodal)) | Yes | Yes | Yes | Yes | Yes | | KV-Aware Routing | Yes | n/a | Yes | Yes | Yes — The Rust frontend supports models handled by `llm-multimodal`; the Python path delegates to vLLM's multimodal processor. ([https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing](https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing)) | Yes | Yes | Yes — vLLM routes requests based on LoRA adapter affinity. | Yes | Yes | | SLA-Based Planner | Yes | Yes | n/a | Yes | n/a | Yes | Yes | n/a | Yes | n/a | | KV Block Manager | Yes | Yes | Yes | n/a | Yes | Yes | Yes | Yes | Yes | Yes | | Multimodal | Yes — Supports Qwen2-Audio experimentally and video input with frame sampling. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/v-llm/vllm-multimodal](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/v-llm/vllm-multimodal)) | Yes — The Rust frontend supports models handled by `llm-multimodal`; the Python path delegates to vLLM's multimodal processor. ([https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing](https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing)) | n/a | Yes | n/a | Yes | Yes | n/a | Yes | n/a | | Request Migration | Yes | Yes | Yes | Yes | Yes | n/a | Yes | Yes | Yes | Yes | | Request Cancellation | Yes | Yes | Yes | Yes | Yes | Yes | n/a | Yes | Yes | Yes | | LoRA | Yes | Yes — vLLM routes requests based on LoRA adapter affinity. | n/a | Yes | n/a | Yes | Yes | n/a | Yes | n/a | | Tool Calling | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | n/a | Yes — Eagle3 support is documented. ([https://docs.nvidia.com/dynamo/dev/additional-resources/speculative-decoding/speculative-decoding-with-v-llm](https://docs.nvidia.com/dynamo/dev/additional-resources/speculative-decoding/speculative-decoding-with-v-llm)) | | Speculative Decoding | Yes | Yes | n/a | Yes | n/a | Yes | Yes | n/a | Yes — Eagle3 support is documented. ([https://docs.nvidia.com/dynamo/dev/additional-resources/speculative-decoding/speculative-decoding-with-v-llm](https://docs.nvidia.com/dynamo/dev/additional-resources/speculative-decoding/speculative-decoding-with-v-llm)) | n/a | *SGLang* | Feature | Disaggregated Serving | KV-Aware Routing | SLA-Based Planner | KV Block Manager | Multimodal | Request Migration | Request Cancellation | LoRA | Tool Calling | Speculative Decoding | | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Disaggregated Serving | n/a | Yes | Yes | Experimental | Yes — Supports aggregated EPD, E/PD, and E/P/D patterns. Traditional disaggregated EP/D is not supported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/sglang-multimodal](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/sglang-multimodal)) | Yes | Experimental — Cancellation during remote prefill is not supported in disaggregated mode. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Experimental — Prefill/decode lifecycle registration has unit coverage, but no SGLang disaggregated LoRA end-to-end test. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Yes | Experimental — Code hooks exist, but examples and documentation are not yet available. | | KV-Aware Routing | Yes | n/a | Yes | Experimental | Yes — Hash forwarding is upstream in SGLang 0.5.13+ and Dynamo pins 0.5.19, so the shipped image routes on image overlap. A custom build without that patch still serves the request but degrades to text-prefix routing. ([https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing](https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing)) | Yes | Yes | Experimental — Aggregated LoRA inference is validated without the KV router; the combined path remains experimental. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Yes | Experimental | | SLA-Based Planner | Yes | Yes | n/a | Experimental | n/a | Yes | Yes | n/a | Yes | n/a | | KV Block Manager | Experimental | Experimental | Experimental | n/a | Experimental | Experimental | Experimental | Experimental — This LoRA feature pairing is not end-to-end validated. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Experimental | Experimental | | Multimodal | Yes — Supports aggregated EPD, E/PD, and E/P/D patterns. Traditional disaggregated EP/D is not supported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/sglang-multimodal](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/sglang-multimodal)) | Yes — Hash forwarding is upstream in SGLang 0.5.13+ and Dynamo pins 0.5.19, so the shipped image routes on image overlap. A custom build without that patch still serves the request but degrades to text-prefix routing. ([https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing](https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing)) | n/a | Experimental | n/a | Yes | Experimental | n/a | Yes | n/a | | Request Migration | Yes | Yes | Yes | Experimental | Yes | n/a | Yes | Experimental — This LoRA feature pairing is not end-to-end validated. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Yes | Experimental | | Request Cancellation | Experimental — Cancellation during remote prefill is not supported in disaggregated mode. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Yes | Yes | Experimental | Experimental | Yes | n/a | Experimental — This LoRA feature pairing is not end-to-end validated. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Yes | n/a | | LoRA | Experimental — Prefill/decode lifecycle registration has unit coverage, but no SGLang disaggregated LoRA end-to-end test. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Experimental — Aggregated LoRA inference is validated without the KV router; the combined path remains experimental. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | n/a | Experimental — This LoRA feature pairing is not end-to-end validated. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | n/a | Experimental — This LoRA feature pairing is not end-to-end validated. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Experimental — This LoRA feature pairing is not end-to-end validated. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | n/a | Experimental — Tool calling with SGLang LoRA is not end-to-end validated. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Experimental — Speculative decoding with SGLang LoRA is not end-to-end validated. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | | Tool Calling | Yes | Yes | Yes | Experimental | Yes | Yes | Yes | Experimental — Tool calling with SGLang LoRA is not end-to-end validated. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | n/a | Experimental | | Speculative Decoding | Experimental — Code hooks exist, but examples and documentation are not yet available. | Experimental | n/a | Experimental | n/a | Experimental | n/a | Experimental — Speculative decoding with SGLang LoRA is not end-to-end validated. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Experimental | n/a | *TensorRT-LLM* | Feature | Disaggregated Serving | KV-Aware Routing | SLA-Based Planner | KV Block Manager | Multimodal | Request Migration | Request Cancellation | LoRA | Tool Calling | Speculative Decoding | | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Disaggregated Serving | n/a | Yes | Yes | Yes | Yes — Supports EP/D and E/P/D image flows with image URLs and pre-computed embeddings. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/tensorrt-llm-multimodal](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/tensorrt-llm-multimodal)) | Yes | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | Yes | Yes | | KV-Aware Routing | Yes | n/a | Yes | Yes | Yes — Workers must publish KV events with block reuse enabled. ([https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing](https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing)) | Yes | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | Yes | Yes | | SLA-Based Planner | Yes | Yes | n/a | Yes | n/a | Yes | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | Yes | n/a | | KV Block Manager | Yes | Yes | Yes | n/a | Yes | Yes | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | Yes | Yes | | Multimodal | Yes — Supports EP/D and E/P/D image flows with image URLs and pre-computed embeddings. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/tensorrt-llm-multimodal](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/tensorrt-llm-multimodal)) | Yes — Workers must publish KV events with block reuse enabled. ([https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing](https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing)) | n/a | Yes | n/a | Experimental | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | Yes | n/a | | Request Migration | Yes | Yes | Yes | Yes | Experimental | n/a | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | Yes | Yes | | Request Cancellation | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | n/a | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | Yes | Yes | | LoRA | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | n/a | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | | Tool Calling | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | n/a | Yes | | Speculative Decoding | Yes | Yes | n/a | Yes | n/a | Yes | Yes | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | Yes | n/a | **Platform support** * GPU architectures: Blackwell, Hopper, Ada Lovelace, Ampere * OS: Ubuntu 24.04 (x86\_64, ARM64) — Containers and wheels * OS: Ubuntu 22.04 (x86\_64) — Wheels only * CSP: AWS — Amazon Linux 2023 (x86\_64) — Containers and wheels * CPU architectures: x86\_64, ARM64 (Ubuntu 24.04 only) * Wheels: Wheels are built in a manylinux\_2\_28 environment (AlmaLinux 8, glibc 2.28+) and validated on Ubuntu 22.04 and 24.04. They install on any Linux distribution with glibc 2.28+ (Debian 11+, RHEL 9, etc.), but only Ubuntu 22.04/24.04 are officially verified.