> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt. # Compatibility The card above covers one release at a time. To compare CUDA toolkit and minimum driver requirements across releases per backend, jump to the [Release Support Matrix](#release-support-matrix) at the bottom of this page. For extended driver compatibility beyond the listed minimums, including forward compatibility and `cuda-compat` packages, see the [CUDA Compatibility documentation](https://docs.nvidia.com/deploy/cuda-compatibility/latest/). See [Release Artifacts](/dynamo/dev/reference/release-artifacts) for the full artifact inventory — container images, wheels, Helm charts, and crates — [Local Installation](/dynamo/dev/cli/installation/install-dynamo) for host OS and architecture requirements, and [Model Early Access Builds](/dynamo/dev/reference/model-early-access-builds) for per-model early access container builds. For backend-specific runtime workarounds — including the local `docker run --network host` hang first reported on Amazon Linux 2023 — see [TensorRT-LLM Known Issues](/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/known-issues). ## Feature Support ### Per-Backend Detail #### vLLM vLLM offers the broadest feature coverage in Dynamo, with full support for disaggregated serving, KV-aware routing, KV block management, LoRA adapters, and multimodal inference including video and audio. *Source: [docs/backends/vllm/README.md][vllm-readme]* | Feature | Supported? | Notes | | :--------------------------------------------------------------------------------------------------------- | :--------: | :---------------------------------------------------------------------------------------------------------------------------------------- | | **Disaggregated Serving** | ✓ | Prefill/decode separation with NIXL KV transfer | | **KV-Aware Routing** | ✓ | | | **SLA-Based Planner** | ✓ | | | **KV Block Manager** | ✓ | | | **Multimodal** | ✓ | Image + video; audio experimental (Qwen2-Audio). With KV-aware routing, image-aware routing on documented paths ([Source][mm-kv-routing]) | | **Request Migration** | ✓ | | | **Request Cancellation** | ✓ | | | **LoRA** | ✓ | Dynamic load/unload; KV-aware routing supports adapter affinity | | **Tool Calling** | ✓ | | | **Speculative Decoding** | ✓ | Eagle3 ([Source][vllm-spec]) | | **[GPU Memory Service](/dynamo/dev/kubernetes/fault-tolerance/shadow-engine-failover#gpu-memory-service)** | ✓ | Weights and KV; upstream integration remains in progress | | **[Shadow Engine Failover](/dynamo/dev/kubernetes/fault-tolerance/shadow-engine-failover)** | ! | Software-process failover only; no KV-cache reuse or hardware fault tolerance | | **[Dynamo Snapshot](/dynamo/dev/kubernetes/operations/cold-start-optimizations/dynamo-snapshot)** | ! | Single-GPU supported; multi-GPU is highly experimental and multinode remains in progress | #### SGLang SGLang is optimized for high-throughput serving with fast primitives, providing robust support for disaggregated serving, KV-aware routing, and request migration. *Source: [docs/backends/sglang/README.md][sglang-readme]* | Feature | Supported? | Notes | | :--------------------------------------------------------------------------------------------------------- | :--------: | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Disaggregated Serving** | ✓ | | | **KV-Aware Routing** | ✓ | | | **SLA-Based Planner** | ✓ | | | **KV Block Manager** | WIP | Work in progress across all combinations | | **Multimodal** | ✓ | Image + video. KV-aware routing supported on Dynamo's SGLang image; a custom build without the hash-forwarding patch falls back to text-prefix routing. Disagg patterns: EPD, E/PD, E/P/D (not traditional EP/D) ([Source][mm-sglang]) | | **Request Migration** | ✓ | | | **Request Cancellation** | WIP | Remote-prefill-phase cancellation not supported in disaggregated mode ([Source][sglang-readme]) | | **LoRA** | — | Not supported | | **Tool Calling** | ✓ | | | **Speculative Decoding** | WIP | Code hooks exist; no examples or docs yet | | **[GPU Memory Service](/dynamo/dev/kubernetes/fault-tolerance/shadow-engine-failover#gpu-memory-service)** | ✓ | Weights and KV; upstream integration remains in progress | | **[Shadow Engine Failover](/dynamo/dev/kubernetes/fault-tolerance/shadow-engine-failover)** | WIP | Experimental; no KV-cache reuse or hardware fault tolerance | | **[Dynamo Snapshot](/dynamo/dev/kubernetes/operations/cold-start-optimizations/dynamo-snapshot)** | ! | Single-GPU supported; multi-GPU and multinode remain in progress | #### TensorRT-LLM TensorRT-LLM delivers maximum inference performance and optimization, with full KVBM integration and robust disaggregated serving support. *Source: [docs/backends/trtllm/README.md][trtllm-readme]* | Feature | Supported? | Notes | | :--------------------------------------------------------------------------------------------------------- | :--------: | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Disaggregated Serving** | ✓ | | | **KV-Aware Routing** | ✓ | | | **SLA-Based Planner** | ✓ | | | **KV Block Manager** | ✓ | | | **Multimodal** | ✓ | Image only (URLs + pre-computed embeddings). Disagg: EP/D + E/P/D. With KV-aware routing, via dedicated MM Router Worker (requires KV event publishing) ([Source][mm-trtllm]) | | **Request Migration** | ✓ | Work in progress with multimodal | | **Request Cancellation** | ! | Engine temporarily not notified of cancellations — resources for cancelled requests are not freed (known issue) | | **LoRA** | — | Not supported | | **Tool Calling** | ✓ | | | **Speculative Decoding** | ✓ | | | **[GPU Memory Service](/dynamo/dev/kubernetes/fault-tolerance/shadow-engine-failover#gpu-memory-service)** | WIP | Weights only; multinode and upstream integration remain in progress | | **[Shadow Engine Failover](/dynamo/dev/kubernetes/fault-tolerance/shadow-engine-failover)** | WIP | Experimental; no KV-cache reuse or hardware fault tolerance | | **[Dynamo Snapshot](/dynamo/dev/kubernetes/operations/cold-start-optimizations/dynamo-snapshot)** | WIP | Experimental single-GPU aggregated text-worker path only | ### Feature Interactions Pairwise feature-by-feature compatibility within each backend. Each cell reports whether the row feature works together with the column feature. #### vLLM #### SGLang #### TensorRT-LLM [vllm-readme]: ../../developer-guide/knowledge-base/modular-components/backends/vllm/overview.md [sglang-readme]: ../../developer-guide/knowledge-base/modular-components/backends/sglang/overview.md [trtllm-readme]: ../../developer-guide/knowledge-base/modular-components/backends/tensorrt-llm/overview.md [mm-kv-routing]: ../../use-cases/multimodal-serving/multimodal-kv-routing.md [mm-sglang]: ../../developer-guide/knowledge-base/modular-components/backends/sglang/multimodal.md [mm-trtllm]: ../../developer-guide/knowledge-base/modular-components/backends/tensorrt-llm/multimodal.md [vllm-spec]: ../../developer-guide/additional-resources/speculative-decoding/speculative-decoding-with-vllm.md ## Release Support Matrix Every stable release and patch, grouped by minor line — newest first. Expand a line to see each release's backend pins, CUDA toolkit, and minimum driver, ordered CUDA 12 before CUDA 13. Releases predating per-release CUDA tracking are listed with their requirements marked "Not recorded" rather than dropped. Platform previews and model-specific builds are excluded; the notes below call out the ones whose toolkit support differs, and the [Releases (machine-readable)](/dynamo/dev/reference/releases-data) page has the full inventory. Driver already installed? Read across from your version — each cell is the newest release that backend can run on it. A driver meeting a higher floor also runs everything below it. | Driver | SGLang | TensorRT-LLM | vLLM | | ------- | ------ | ------------ | ----------- | | 570.xx+ | 0.7.1 | None | 0.7.0.post1 | | 575.xx+ | 1.2.1 | None | 1.2.1 | | 580.xx+ | 1.3.1 | 1.3.1 | 1.3.1 | Current stable release: v1.3.1 (container tag `1.3.1`, wheel version `1.3.1`). **Backend engine pins per Dynamo release** | Dynamo | Type | SGLang | TensorRT-LLM | vLLM | NIXL (SGL / TRT / vLLM) | UCX | | ------------------------ | ---------------- | --------------------- | -------------- | ------ | ------------------------ | ------ | | main (ToT) | development head | 0.5.16 | 1.3.0rc23 | 0.26.0 | 1.3.0 / 1.3.1 / 1.3.2 | - | | v1.3.1 | patch | 0.5.14 | 1.3.0rc19 | 0.23.0 | 1.3.2 / 1.0.1 / 1.1.0 | 1.20.x | | v1.3.0 | stable | 0.5.14 | 1.3.0rc19 | 0.23.0 | 1.3.0 / 1.0.1 / 1.1.0 | 1.20.x | | v1.3.0-dev.1 | platform-preview | 0.5.12.post1 | 1.3.0rc17 | 0.22.0 | 1.0.1 / 0.10.1 / 1.1.0 | - | | v1.2.1 | patch | 0.5.11 | 1.3.0rc14 | 0.20.1 | 1.0.1 / 0.10.1 / 0.10.1 | - | | v1.2.0 | stable | 0.5.11 | 1.3.0rc14 | 0.20.1 | 1.0.1 / 0.10.1 / 0.10.1 | 1.20.0 | | v1.2.0-deepseek-v4-dev.3 | model-build | upstream DSv4 preview | - | 0.20.1 | - / - / 0.10.1 | - | | v1.2.0-deepseek-v4-dev.2 | model-build | upstream DSv4 preview | - | 0.20.0 | - / - / 0.10.1 | - | | v1.1.1 | patch | 0.5.10.post1 | 1.3.0rc11 | 0.19.0 | 1.0.1 / 0.10.1 / 0.10.1 | - | | v1.1.0 | stable | 0.5.10.post1 | 1.3.0rc11 | 0.19.0 | 1.0.1 / 0.10.1 / 0.10.1 | 1.20 | | v1.1.0-dev.3 | platform-preview | 0.5.10.post1 | 1.3.0rc11 | 0.19.0 | 1.0.1 / 0.10.1 / 0.10.1 | - | | v1.1.0-dev.2 | platform-preview | 0.5.9 | 1.3.0rc9 | 0.19.0 | 1.0.1 / 0.10.1 / 0.10.1 | - | | v1.1.0-dev.1 | platform-preview | 0.5.9 | 1.3.0rc5.post1 | 0.17.1 | 1.0.1 / 0.10.1 / 0.10.1 | - | | v1.0.2 | patch | 0.5.9 | 1.3.0rc5.post1 | 0.16.0 | 0.10.1 / 0.10.1 / 0.10.1 | - | | v1.0.1 | patch | 0.5.9 | 1.3.0rc5.post1 | 0.16.0 | 0.10.1 / 0.10.1 / 0.10.1 | - | | v1.0.0 | stable | 0.5.9 | 1.3.0rc5.post1 | 0.16.0 | 0.10.1 / 0.10.1 / 0.10.1 | - | | v0.9.1 | patch | 0.5.8 | 1.3.0rc3 | 0.14.1 | 0.9.0 / 0.9.0 / 0.9.0 | - | | v0.9.0 | stable | 0.5.8 | 1.3.0rc1 | 0.14.1 | 0.9.0 / 0.9.0 / 0.9.0 | - | | v0.8.1.post3 | patch | 0.5.6.post2 | 1.2.0rc6.post3 | 0.12.0 | 0.8.0 / 0.8.0 / 0.8.0 | - | | v0.8.1.post2 | patch | 0.5.6.post2 | 1.2.0rc6.post2 | 0.12.0 | 0.8.0 / 0.8.0 / 0.8.0 | - | | v0.8.1.post1 | patch | 0.5.6.post2 | 1.2.0rc6.post1 | 0.12.0 | 0.8.0 / 0.8.0 / 0.8.0 | - | | v0.8.1 | patch | 0.5.6.post2 | 1.2.0rc6.post1 | 0.12.0 | 0.8.0 / 0.8.0 / 0.8.0 | - | | v0.8.0 | stable | 0.5.6.post2 | 1.2.0rc6.post1 | 0.12.0 | 0.8.0 / 0.8.0 / 0.8.0 | - | | v0.7.1 | patch | 0.5.4.post3 | 1.2.0rc3 | 0.11.0 | 0.8.0 / 0.8.0 / 0.8.0 | - | | v0.7.0.post1 | patch | 0.5.4.post3 | 1.2.0rc3 | 0.11.0 | 0.8.0 / 0.8.0 / 0.8.0 | - | | v0.7.0 | stable | 0.5.4.post3 | 1.2.0rc2 | 0.11.0 | 0.8.0 / 0.8.0 / 0.8.0 | - | | v0.6.1.post1 | patch | 0.5.3.post2 | 1.1.0rc5 | 0.11.0 | 0.6.0 / 0.6.0 / 0.6.0 | - | | v0.6.1 | patch | 0.5.3.post2 | 1.1.0rc5 | 0.11.0 | 0.6.0 / 0.6.0 / 0.6.0 | - | | v0.6.0 | stable | 0.5.3.post2 | 1.1.0rc5 | 0.11.0 | 0.6.0 / 0.6.0 / 0.6.0 | - | **CUDA toolkit and minimum driver per Dynamo release** | Dynamo | Backend | CUDA Toolkit | Min Driver | Note | | ------ | ------------ | ------------ | ---------- | ------------ | | 1.3.1 | SGLang | 13.0 | 580.xx+ | - | | 1.3.1 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.3.1 | vLLM | 13.0 | 580.xx+ | - | | 1.3.0 | SGLang | 13.0 | 580.xx+ | - | | 1.3.0 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.3.0 | vLLM | 13.0 | 580.xx+ | - | | 1.2.1 | SGLang | 12.9 | 575.xx+ | - | | 1.2.1 | SGLang | 13.0 | 580.xx+ | - | | 1.2.1 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.2.1 | vLLM | 12.9 | 575.xx+ | - | | 1.2.1 | vLLM | 13.0 | 580.xx+ | - | | 1.2.0 | SGLang | 12.9 | 575.xx+ | - | | 1.2.0 | SGLang | 13.0 | 580.xx+ | - | | 1.2.0 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.2.0 | vLLM | 12.9 | 575.xx+ | - | | 1.2.0 | vLLM | 13.0 | 580.xx+ | - | | 1.1.1 | SGLang | 12.9 | 575.xx+ | - | | 1.1.1 | SGLang | 13.0 | 580.xx+ | - | | 1.1.1 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.1.1 | vLLM | 12.9 | 575.xx+ | - | | 1.1.1 | vLLM | 13.0 | 580.xx+ | - | | 1.1.0 | SGLang | 12.9 | 575.xx+ | - | | 1.1.0 | SGLang | 13.0 | 580.xx+ | - | | 1.1.0 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.1.0 | vLLM | 12.9 | 575.xx+ | - | | 1.1.0 | vLLM | 13.0 | 580.xx+ | - | | 1.0.2 | SGLang | 12.9 | 575.xx+ | - | | 1.0.2 | SGLang | 13.0 | 580.xx+ | - | | 1.0.2 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.0.2 | vLLM | 12.9 | 575.xx+ | - | | 1.0.2 | vLLM | 13.0 | 580.xx+ | - | | 1.0.1 | SGLang | 12.9 | 575.xx+ | - | | 1.0.1 | SGLang | 13.0 | 580.xx+ | - | | 1.0.1 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.0.1 | vLLM | 12.9 | 575.xx+ | - | | 1.0.1 | vLLM | 13.0 | 580.xx+ | - | | 1.0.0 | SGLang | 12.9 | 575.xx+ | - | | 1.0.0 | SGLang | 13.0 | 580.xx+ | - | | 1.0.0 | TensorRT-LLM | 13.1 | 580.xx+ | - | | 1.0.0 | vLLM | 12.9 | 575.xx+ | - | | 1.0.0 | vLLM | 13.0 | 580.xx+ | - | | 0.9.1 | SGLang | 12.9 | 575.xx+ | - | | 0.9.1 | TensorRT-LLM | 13.0 | 580.xx+ | - | | 0.9.1 | vLLM | 12.9 | 575.xx+ | - | | 0.9.0 | SGLang | 12.9 | 575.xx+ | - | | 0.9.0 | TensorRT-LLM | 13.0 | 580.xx+ | - | | 0.9.0 | vLLM | 12.9 | 575.xx+ | - | | 0.8.1 | SGLang | 12.9 | 575.xx+ | - | | 0.8.1 | SGLang | 13.0 | 580.xx+ | Experimental | | 0.8.1 | TensorRT-LLM | 13.0 | 580.xx+ | - | | 0.8.1 | vLLM | 12.9 | 575.xx+ | - | | 0.8.1 | vLLM | 13.0 | 580.xx+ | Experimental | | 0.8.0 | SGLang | 12.9 | 575.xx+ | - | | 0.8.0 | SGLang | 13.0 | 580.xx+ | Experimental | | 0.8.0 | TensorRT-LLM | 13.0 | 580.xx+ | - | | 0.8.0 | vLLM | 12.9 | 575.xx+ | - | | 0.8.0 | vLLM | 13.0 | 580.xx+ | Experimental | | 0.7.1 | SGLang | 12.8 | 570.xx+ | - | | 0.7.1 | TensorRT-LLM | 13.0 | 580.xx+ | - | | 0.7.1 | vLLM | 12.9 | 575.xx+ | - | | 0.7.0 | SGLang | 12.9 | 575.xx+ | - | | 0.7.0 | TensorRT-LLM | 13.0 | 580.xx+ | - | | 0.7.0 | vLLM | 12.8 | 570.xx+ | - | * Patch versions (e.g. v0.8.1.post1, v0.7.0.post1) have the same CUDA support as their base version. * Early access v1.1.0-dev.\* images follow the same CUDA matrix as v1.0.2. The v1.2.0-deepseek-v4-dev.3 vLLM container is CUDA 13.0 multi-arch; the SGLang containers split by arch (CUDA 12.9 on amd64, CUDA 13.0 on arm64). * Experimental CUDA 13 images are not published for all versions. **Feature support by backend (v1.3.1)** | Feature | SGLang | TensorRT-LLM | vLLM | | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------- | | Disaggregated Serving | Supported | Supported | Supported (Prefill/decode separation with NIXL KV transfer) | | KV-Aware Routing | Supported | Supported | Supported | | SLA-Based Planner | Supported | Supported | Supported | | KV Block Manager | Experimental (Work in progress across all combinations) | Supported | Supported | | Multimodal (Image) | Supported (KV-aware routing supported on Dynamo's SGLang image for aggregated workers; a custom build without the hash-forwarding patch falls back to text-prefix routing. Separately, multimodal serving supports EPD, E/PD and E/P/D disaggregation (not traditional EP/D)) | Supported (Image URLs + pre-computed embeddings. Disagg: EP/D + E/P/D. KV-aware routing via dedicated MM Router Worker (requires KV event publishing)) | Supported (With KV-aware routing, image-aware routing on documented paths) | | Multimodal (Video) | Supported | Not supported | Supported (Video input with frame sampling) | | Multimodal (Audio) | Not supported | Not supported | Experimental (Qwen2-Audio, experimental) | | Request Migration | Supported | Supported (Work in progress with multimodal) | Supported | | Request Cancellation | Experimental (Remote-prefill-phase cancellation not supported in disaggregated mode) | Supported with caveat (Engine temporarily not notified of cancellations — resources for cancelled requests are not freed (known issue)) | Supported | | LoRA | Not supported | Not supported | Supported (Dynamic load/unload; KV-aware routing supports adapter affinity) | | Tool Calling | Supported | Supported | Supported | | Speculative Decoding | Experimental (Code hooks exist; no examples or docs yet) | Supported | Supported (Eagle3) | | GPU Memory Service | Supported (Weights and KV; upstream integration remains in progress) | Experimental (Weights only; multinode and upstream integration remain in progress) | Supported (Weights and KV; upstream integration remains in progress) | | Shadow Engine Failover | Experimental (No KV-cache reuse or hardware fault tolerance) | Experimental (No KV-cache reuse or hardware fault tolerance) | Supported with caveat (Software-process failover only; no KV-cache reuse or hardware fault tolerance) | | Dynamo Snapshot | Supported with caveat (Single-GPU supported; multi-GPU and multinode remain in progress) | Experimental (Single-GPU aggregated text-worker path only) | Supported with caveat (Single-GPU supported; multi-GPU is highly experimental and multinode remains in progress) | **Feature interactions by backend** Each cell states whether the row feature works together with the column feature. The matrix is symmetric: a cell reads the same in either direction. *vLLM* | Feature | Disaggregated Serving | KV-Aware Routing | SLA-Based Planner | KV Block Manager | Multimodal | Request Migration | Request Cancellation | LoRA | Tool Calling | Speculative Decoding | | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- | -------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Disaggregated Serving | n/a | Yes | Yes | Yes | Yes — Supports Qwen2-Audio experimentally and video input with frame sampling. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/v-llm/vllm-multimodal](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/v-llm/vllm-multimodal)) | Yes | Yes | Yes | Yes | Yes | | KV-Aware Routing | Yes | n/a | Yes | Yes | Yes — The Rust frontend supports models handled by `llm-multimodal`; the Python path delegates to vLLM's multimodal processor. ([https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing](https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing)) | Yes | Yes | Yes — vLLM routes requests based on LoRA adapter affinity. | Yes | Yes | | SLA-Based Planner | Yes | Yes | n/a | Yes | n/a | Yes | Yes | n/a | Yes | n/a | | KV Block Manager | Yes | Yes | Yes | n/a | Yes | Yes | Yes | Yes | Yes | Yes | | Multimodal | Yes — Supports Qwen2-Audio experimentally and video input with frame sampling. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/v-llm/vllm-multimodal](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/v-llm/vllm-multimodal)) | Yes — The Rust frontend supports models handled by `llm-multimodal`; the Python path delegates to vLLM's multimodal processor. ([https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing](https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing)) | n/a | Yes | n/a | Yes | Yes | n/a | Yes | n/a | | Request Migration | Yes | Yes | Yes | Yes | Yes | n/a | Yes | Yes | Yes | Yes | | Request Cancellation | Yes | Yes | Yes | Yes | Yes | Yes | n/a | Yes | Yes | Yes | | LoRA | Yes | Yes — vLLM routes requests based on LoRA adapter affinity. | n/a | Yes | n/a | Yes | Yes | n/a | Yes | n/a | | Tool Calling | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | n/a | Yes — Eagle3 support is documented. ([https://docs.nvidia.com/dynamo/dev/additional-resources/speculative-decoding/speculative-decoding-with-v-llm](https://docs.nvidia.com/dynamo/dev/additional-resources/speculative-decoding/speculative-decoding-with-v-llm)) | | Speculative Decoding | Yes | Yes | n/a | Yes | n/a | Yes | Yes | n/a | Yes — Eagle3 support is documented. ([https://docs.nvidia.com/dynamo/dev/additional-resources/speculative-decoding/speculative-decoding-with-v-llm](https://docs.nvidia.com/dynamo/dev/additional-resources/speculative-decoding/speculative-decoding-with-v-llm)) | n/a | *SGLang* | Feature | Disaggregated Serving | KV-Aware Routing | SLA-Based Planner | KV Block Manager | Multimodal | Request Migration | Request Cancellation | LoRA | Tool Calling | Speculative Decoding | | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Disaggregated Serving | n/a | Yes | Yes | Experimental | Yes — Supports aggregated EPD, E/PD, and E/P/D patterns. Traditional disaggregated EP/D is not supported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/sglang-multimodal](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/sglang-multimodal)) | Yes | Experimental — Cancellation during remote prefill is not supported in disaggregated mode. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | No — SGLang does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Yes | Experimental — Code hooks exist, but examples and documentation are not yet available. | | KV-Aware Routing | Yes | n/a | Yes | Experimental | Yes — Hash forwarding is upstream in SGLang 0.5.13+ and Dynamo pins 0.5.16, so the shipped image routes on image overlap. A custom build without that patch still serves the request but degrades to text-prefix routing. ([https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing](https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing)) | Yes | Yes | No — SGLang does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Yes | Experimental | | SLA-Based Planner | Yes | Yes | n/a | Experimental | n/a | Yes | Yes | No — SGLang does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Yes | n/a | | KV Block Manager | Experimental | Experimental | Experimental | n/a | Experimental | Experimental | Experimental | No — SGLang does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Experimental | Experimental | | Multimodal | Yes — Supports aggregated EPD, E/PD, and E/P/D patterns. Traditional disaggregated EP/D is not supported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/sglang-multimodal](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/sglang-multimodal)) | Yes — Hash forwarding is upstream in SGLang 0.5.13+ and Dynamo pins 0.5.16, so the shipped image routes on image overlap. A custom build without that patch still serves the request but degrades to text-prefix routing. ([https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing](https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing)) | n/a | Experimental | n/a | Yes | Experimental | No — SGLang does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Yes | n/a | | Request Migration | Yes | Yes | Yes | Experimental | Yes | n/a | Yes | No — SGLang does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Yes | Experimental | | Request Cancellation | Experimental — Cancellation during remote prefill is not supported in disaggregated mode. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Yes | Yes | Experimental | Experimental | Yes | n/a | No — SGLang does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Yes | n/a | | LoRA | No — SGLang does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | No — SGLang does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | No — SGLang does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | No — SGLang does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | No — SGLang does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | No — SGLang does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | No — SGLang does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | n/a | No — SGLang does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | No — SGLang does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | | Tool Calling | Yes | Yes | Yes | Experimental | Yes | Yes | Yes | No — SGLang does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | n/a | Experimental | | Speculative Decoding | Experimental — Code hooks exist, but examples and documentation are not yet available. | Experimental | n/a | Experimental | n/a | Experimental | n/a | No — SGLang does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview)) | Experimental | n/a | *TensorRT-LLM* | Feature | Disaggregated Serving | KV-Aware Routing | SLA-Based Planner | KV Block Manager | Multimodal | Request Migration | Request Cancellation | LoRA | Tool Calling | Speculative Decoding | | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Disaggregated Serving | n/a | Yes | Yes | Yes | Yes — Supports EP/D and E/P/D image flows with image URLs and pre-computed embeddings. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/tensorrt-llm-multimodal](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/tensorrt-llm-multimodal)) | Yes | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | Yes | Yes | | KV-Aware Routing | Yes | n/a | Yes | Yes | Yes — Workers must publish KV events with block reuse enabled. ([https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing](https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing)) | Yes | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | Yes | Yes | | SLA-Based Planner | Yes | Yes | n/a | Yes | n/a | Yes | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | Yes | n/a | | KV Block Manager | Yes | Yes | Yes | n/a | Yes | Yes | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | Yes | Yes | | Multimodal | Yes — Supports EP/D and E/P/D image flows with image URLs and pre-computed embeddings. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/tensorrt-llm-multimodal](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/tensorrt-llm-multimodal)) | Yes — Workers must publish KV events with block reuse enabled. ([https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing](https://docs.nvidia.com/dynamo/dev/multimodal/multimodal-kv-routing)) | n/a | Yes | n/a | Experimental | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | Yes | n/a | | Request Migration | Yes | Yes | Yes | Yes | Experimental | n/a | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | Yes | Yes | | Request Cancellation | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | Yes — The engine is temporarily not notified of cancellations, so resources for cancelled requests are not freed. | n/a | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | Yes | Yes | | LoRA | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | n/a | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | | Tool Calling | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | n/a | Yes | | Speculative Decoding | Yes | Yes | n/a | Yes | n/a | Yes | Yes | No — TensorRT-LLM does not support LoRA in Dynamo, so every LoRA pairing is unsupported. ([https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview](https://docs.nvidia.com/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview)) | Yes | n/a | **Platform support** * GPU architectures: Blackwell, Hopper, Ada Lovelace, Ampere * OS: Ubuntu 24.04 (x86\_64, ARM64) — Containers and wheels * OS: Ubuntu 22.04 (x86\_64) — Wheels only * CSP: AWS — Amazon Linux 2023 (x86\_64) — Containers and wheels * CPU architectures: x86\_64, ARM64 (Ubuntu 24.04 only) * Wheels: Wheels are built in a manylinux\_2\_28 environment (AlmaLinux 8, glibc 2.28+) and validated on Ubuntu 22.04 and 24.04. They install on any Linux distribution with glibc 2.28+ (Debian 11+, RHEL 9, etc.), but only Ubuntu 22.04/24.04 are officially verified.