> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.
# Compatibility
For extended driver compatibility beyond the listed minimums, including forward compatibility and `cuda-compat` packages, see the [CUDA Compatibility documentation](https://docs.nvidia.com/deploy/cuda-compatibility/).
See [Release Artifacts](/dynamo/dev/reference/release-artifacts) for the full artifact inventory — container images, wheels, Helm charts, and crates — and [Model Early Access Builds](/dynamo/dev/reference/model-early-access-builds) for per-model early access container builds.
## Find a Compatible Release
Pick your backend and the CUDA generation your host driver supports to see which Dynamo releases you can run — and what to pull for the current one.
## Platform Notes
Dynamo ships multi-arch (x86\_64 + ARM64) container images. Wheels are built in a manylinux\_2\_28-compatible environment and validated on CentOS Stream 9 and Ubuntu 22.04/24.04; other Linux distributions are expected to work but are not officially verified.
### Cloud Service Providers
**Amazon Linux 2023** (AWS) · x86\_64 · Supported
**AL2023 TensorRT-LLM limitation:** there is a known issue with the TensorRT-LLM framework when running the AL2023 container locally with `docker run --network host ...` due to a [bug](https://github.com/mpi4py/mpi4py/discussions/491#discussioncomment-12660609) in mpi4py. Replace the `--network host` flag with precise networking configuration by mapping only the necessary ports (4222 for NATS, 2379/2380 for etcd, 8000 for the frontend).
## Feature Support
### Per-Backend Detail
#### vLLM
vLLM offers the broadest feature coverage in Dynamo, with full support for disaggregated serving, KV-aware routing, KV block management, LoRA adapters, and multimodal inference including video and audio.
*Source: [docs/backends/vllm/README.md][vllm-readme]*
| Feature | Supported? | Notes |
| :------------------------ | :--------: | :---------------------------------------------------------------------------------------------------------------------------------------- |
| **Disaggregated Serving** | ✓ | Prefill/decode separation with NIXL KV transfer |
| **KV-Aware Routing** | ✓ | |
| **SLA-Based Planner** | ✓ | |
| **KV Block Manager** | ✓ | |
| **Multimodal** | ✓ | Image + video; audio experimental (Qwen2-Audio). With KV-aware routing, image-aware routing on documented paths ([Source][mm-kv-routing]) |
| **Request Migration** | ✓ | |
| **Request Cancellation** | ✓ | |
| **LoRA** | ✓ | Dynamic load/unload; KV-aware routing supports adapter affinity |
| **Tool Calling** | ✓ | |
| **Speculative Decoding** | ✓ | Eagle3 ([Source][vllm-spec]) |
#### SGLang
SGLang is optimized for high-throughput serving with fast primitives, providing robust support for disaggregated serving, KV-aware routing, and request migration.
*Source: [docs/backends/sglang/README.md][sglang-readme]*
| Feature | Supported? | Notes |
| :------------------------ | :--------: | :---------------------------------------------------------------------------------------------------------------------------------- |
| **Disaggregated Serving** | ✓ | |
| **KV-Aware Routing** | ✓ | |
| **SLA-Based Planner** | ✓ | |
| **KV Block Manager** | WIP | Work in progress across all combinations |
| **Multimodal** | ✓ | Image + video. Not compatible with KV-aware routing. Disagg patterns: EPD, E/PD, E/P/D (not traditional EP/D) ([Source][mm-sglang]) |
| **Request Migration** | ✓ | |
| **Request Cancellation** | WIP | Remote-prefill-phase cancellation not supported in disaggregated mode ([Source][sglang-readme]) |
| **LoRA** | — | Not supported |
| **Tool Calling** | ✓ | |
| **Speculative Decoding** | WIP | Code hooks exist; no examples or docs yet |
#### TensorRT-LLM
TensorRT-LLM delivers maximum inference performance and optimization, with full KVBM integration and robust disaggregated serving support.
*Source: [docs/backends/trtllm/README.md][trtllm-readme]*
| Feature | Supported? | Notes |
| :------------------------ | :--------: | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Disaggregated Serving** | ✓ | |
| **KV-Aware Routing** | ✓ | |
| **SLA-Based Planner** | ✓ | |
| **KV Block Manager** | ✓ | |
| **Multimodal** | ✓ | Image only (URLs + pre-computed embeddings). Disagg: EP/D + E/P/D. With KV-aware routing, via dedicated MM Router Worker (requires KV event publishing) ([Source][mm-trtllm]) |
| **Request Migration** | ✓ | Work in progress with multimodal |
| **Request Cancellation** | ! | Engine temporarily not notified of cancellations — resources for cancelled requests are not freed (known issue) |
| **LoRA** | — | Not supported |
| **Tool Calling** | ✓ | |
| **Speculative Decoding** | ✓ | |
### Feature Interactions
Pairwise feature-by-feature compatibility within each backend. Each cell reports whether the row feature works together with the column feature. A `—` marks the diagonal or a combination that does not apply; blank cells are the mirror of the populated lower triangle.
**Legend:** ✓ Supported · WIP Work in Progress / Experimental / Limited
#### vLLM Feature Interactions
| Feature | Disaggregated Serving | KV-Aware Routing | SLA-Based Planner | KV Block Manager | Multimodal | Request Migration | Request Cancellation | LoRA | Tool Calling | Speculative Decoding |
| :------------------------ | :-------------------: | :--------------: | :---------------: | :--------------: | :--------: | :---------------: | :------------------: | :--: | :----------: | :------------------: |
| **Disaggregated Serving** | — | | | | | | | | | |
| **KV-Aware Routing** | ✅ | — | | | | | | | | |
| **SLA-Based Planner** | ✅ | ✅ | — | | | | | | | |
| **KV Block Manager** | ✅ | ✅ | ✅ | — | | | | | | |
| **Multimodal** | ✅ | ✅1 | — | ✅ | — | | | | | |
| **Request Migration** | ✅ | ✅ | ✅ | ✅ | ✅ | — | | | | |
| **Request Cancellation** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | | | |
| **LoRA** | ✅ | ✅2 | — | ✅ | — | ✅ | ✅ | — | | |
| **Tool Calling** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | |
| **Speculative Decoding** | ✅ | ✅ | — | ✅ | — | ✅ | ✅ | — | ✅ | — |
> **Notes:**
>
> 1. **Multimodal + KV-Aware Routing**: Image-aware KV routing is supported in the documented vLLM paths. The default Rust frontend path supports model families handled by `llm-multimodal`; the Python chat-processor path delegates to vLLM's multimodal processor. ([Source][mm-kv-routing])
> 2. **KV-Aware LoRA Routing**: vLLM supports routing requests based on LoRA adapter affinity.
> 3. **Audio Support**: vLLM supports audio models like Qwen2-Audio (experimental). ([Source][mm-vllm])
> 4. **Video Support**: vLLM supports video input with frame sampling. ([Source][mm-vllm])
> 5. **Speculative Decoding**: Eagle3 support documented. ([Source][vllm-spec])
#### SGLang Feature Interactions
| Feature | Disaggregated Serving | KV-Aware Routing | SLA-Based Planner | KV Block Manager | Multimodal | Request Migration | Request Cancellation | LoRA | Tool Calling | Speculative Decoding |
| :------------------------ | :-------------------: | :--------------: | :---------------: | :--------------: | :--------: | :---------------: | :------------------: | :--: | :----------: | :------------------: |
| **Disaggregated Serving** | — | | | | | | | | | |
| **KV-Aware Routing** | ✅ | — | | | | | | | | |
| **SLA-Based Planner** | ✅ | ✅ | — | | | | | | | |
| **KV Block Manager** | 🚧 | 🚧 | 🚧 | — | | | | | | |
| **Multimodal** | ✅2 | 1 | — | 🚧 | — | | | | | |
| **Request Migration** | ✅ | ✅ | ✅ | 🚧 | ✅ | — | | | | |
| **Request Cancellation** | 🚧3 | ✅ | ✅ | 🚧 | 🚧 | ✅ | — | | | |
| **LoRA** | | | | 🚧 | | | | — | | |
| **Tool Calling** | ✅ | ✅ | ✅ | 🚧 | ✅ | ✅ | ✅ | | — | |
| **Speculative Decoding** | 🚧 | 🚧 | — | 🚧 | — | 🚧 | — | | 🚧 | — |
> **Notes:**
>
> 1. **Multimodal + KV-Aware Routing**: Not supported. ([Source][kv-routing])
> 2. **Multimodal Patterns**: Supports simple Aggregated **EPD**, **E/PD**, and **E/P/D** patterns. Traditional Disagg **EP/D** is not supported. ([Source][mm-sglang])
> 3. **Request Cancellation**: Cancellation during the remote prefill phase is not supported in disaggregated mode. ([Source][sglang-readme])
> 4. **Speculative Decoding**: Code hooks exist (`spec_decode_stats` in publisher), but no examples or documentation yet.
#### TensorRT-LLM Feature Interactions
| Feature | Disaggregated Serving | KV-Aware Routing | SLA-Based Planner | KV Block Manager | Multimodal | Request Migration | Request Cancellation | LoRA | Tool Calling | Speculative Decoding |
| :------------------------ | :-------------------: | :--------------: | :---------------: | :--------------: | :-----------: | :---------------: | :------------------: | :--: | :----------: | :------------------: |
| **Disaggregated Serving** | — | | | | | | | | | |
| **KV-Aware Routing** | ✅ | — | | | | | | | | |
| **SLA-Based Planner** | ✅ | ✅ | — | | | | | | | |
| **KV Block Manager** | ✅ | ✅ | ✅ | — | | | | | | |
| **Multimodal** | ✅1 | ✅2 | — | ✅ | — | | | | | |
| **Request Migration** | ✅ | ✅ | ✅ | ✅ | 🚧 | — | | | | |
| **Request Cancellation** | ✅3 | ✅3 | ✅3 | ✅3 | ✅3 | ✅3 | — | | | |
| **LoRA** | | | | | | | | — | | |
| **Tool Calling** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | — | |
| **Speculative Decoding** | ✅ | ✅ | — | ✅ | — | ✅ | ✅ | | ✅ | — |
> **Notes:**
>
> 1. **Multimodal Disaggregation**: Supports **EP/D** (Traditional) and **E/P/D** (Full Disaggregation) image flows, including image URLs and pre-computed embeddings. ([Source][mm-trtllm])
> 2. **Multimodal + KV-Aware Routing**: The native Rust frontend routes supported models using image-aware KV overlap. TRT-LLM workers must publish KV events with block reuse enabled. ([Source][mm-kv-routing])
> 3. **Request Cancellation**: Due to known issues, the TensorRT-LLM engine is temporarily not notified of request cancellations, meaning allocated resources for cancelled requests are not freed.
[vllm-readme]: ../backends/vllm/README.md
[sglang-readme]: ../backends/sglang/README.md
[trtllm-readme]: ../backends/trtllm/README.md
[mm-kv-routing]: ../features/multimodal/multimodal-kv-routing.md
[mm-sglang]: ../features/multimodal/multimodal-sglang.md
[mm-trtllm]: ../features/multimodal/multimodal-trtllm.md
[mm-vllm]: ../features/multimodal/multimodal-vllm.md
[kv-routing]: ../user-guides/kv-cache-aware-routing.md
[vllm-spec]: ../features/speculative-decoding/speculative-decoding-vllm.md
Current stable release: v1.3.0 (container tag `1.3.0`, wheel version `1.3.0.post1`).
**Backend engine pins per Dynamo release**
| Dynamo | Type | SGLang | TensorRT-LLM | vLLM | NIXL (SGL / TRT / vLLM) | UCX |
| ------------------------ | ---------------- | --------------------- | -------------- | ------ | ------------------------ | ------ |
| main (ToT) | development head | 0.5.15 | 1.3.0rc21 | 0.25.1 | 1.3.0 / 1.0.1 / 1.1.0 | - |
| v1.3.0 | stable | 0.5.14 | 1.3.0rc19 | 0.23.0 | 1.3.0 / 1.0.1 / 1.1.0 | 1.20.x |
| v1.3.0-dev.1 | platform-preview | 0.5.12.post1 | 1.3.0rc17 | 0.22.0 | 1.0.1 / 0.10.1 / 1.1.0 | - |
| v1.2.1 | patch | 0.5.11 | 1.3.0rc14 | 0.20.1 | 1.0.1 / 0.10.1 / 0.10.1 | - |
| v1.2.0 | stable | 0.5.11 | 1.3.0rc14 | 0.20.1 | 1.0.1 / 0.10.1 / 0.10.1 | 1.20.0 |
| v1.2.0-deepseek-v4-dev.3 | model-build | upstream DSv4 preview | - | 0.20.1 | - / - / 0.10.1 | - |
| v1.2.0-deepseek-v4-dev.2 | model-build | upstream DSv4 preview | - | 0.20.0 | - / - / 0.10.1 | - |
| v1.1.1 | patch | 0.5.10.post1 | 1.3.0rc11 | 0.19.0 | 1.0.1 / 0.10.1 / 0.10.1 | - |
| v1.1.0 | stable | 0.5.10.post1 | 1.3.0rc11 | 0.19.0 | 1.0.1 / 0.10.1 / 0.10.1 | 1.20 |
| v1.1.0-dev.3 | platform-preview | 0.5.10.post1 | 1.3.0rc11 | 0.19.0 | 1.0.1 / 0.10.1 / 0.10.1 | - |
| v1.1.0-dev.2 | platform-preview | 0.5.9 | 1.3.0rc9 | 0.19.0 | 1.0.1 / 0.10.1 / 0.10.1 | - |
| v1.1.0-dev.1 | platform-preview | 0.5.9 | 1.3.0rc5.post1 | 0.17.1 | 1.0.1 / 0.10.1 / 0.10.1 | - |
| v1.0.2 | patch | 0.5.9 | 1.3.0rc5.post1 | 0.16.0 | 0.10.1 / 0.10.1 / 0.10.1 | - |
| v1.0.1 | patch | 0.5.9 | 1.3.0rc5.post1 | 0.16.0 | 0.10.1 / 0.10.1 / 0.10.1 | - |
| v1.0.0 | stable | 0.5.9 | 1.3.0rc5.post1 | 0.16.0 | 0.10.1 / 0.10.1 / 0.10.1 | - |
| v0.9.1 | patch | 0.5.8 | 1.3.0rc3 | 0.14.1 | 0.9.0 / 0.9.0 / 0.9.0 | - |
| v0.9.0 | stable | 0.5.8 | 1.3.0rc1 | 0.14.1 | 0.9.0 / 0.9.0 / 0.9.0 | - |
| v0.8.1.post3 | patch | 0.5.6.post2 | 1.2.0rc6.post3 | 0.12.0 | 0.8.0 / 0.8.0 / 0.8.0 | - |
| v0.8.1.post2 | patch | 0.5.6.post2 | 1.2.0rc6.post2 | 0.12.0 | 0.8.0 / 0.8.0 / 0.8.0 | - |
| v0.8.1.post1 | patch | 0.5.6.post2 | 1.2.0rc6.post1 | 0.12.0 | 0.8.0 / 0.8.0 / 0.8.0 | - |
| v0.8.1 | patch | 0.5.6.post2 | 1.2.0rc6.post1 | 0.12.0 | 0.8.0 / 0.8.0 / 0.8.0 | - |
| v0.8.0 | stable | 0.5.6.post2 | 1.2.0rc6.post1 | 0.12.0 | 0.8.0 / 0.8.0 / 0.8.0 | - |
| v0.7.1 | patch | 0.5.4.post3 | 1.2.0rc3 | 0.11.0 | 0.8.0 / 0.8.0 / 0.8.0 | - |
| v0.7.0.post1 | patch | 0.5.4.post3 | 1.2.0rc3 | 0.11.0 | 0.8.0 / 0.8.0 / 0.8.0 | - |
| v0.7.0 | stable | 0.5.4.post3 | 1.2.0rc2 | 0.11.0 | 0.8.0 / 0.8.0 / 0.8.0 | - |
| v0.6.1.post1 | patch | 0.5.3.post2 | 1.1.0rc5 | 0.11.0 | 0.6.0 / 0.6.0 / 0.6.0 | - |
| v0.6.1 | patch | 0.5.3.post2 | 1.1.0rc5 | 0.11.0 | 0.6.0 / 0.6.0 / 0.6.0 | - |
| v0.6.0 | stable | 0.5.3.post2 | 1.1.0rc5 | 0.11.0 | 0.6.0 / 0.6.0 / 0.6.0 | - |
**CUDA toolkit and minimum driver per Dynamo release**
| Dynamo | Backend | CUDA Toolkit | Min Driver | Note |
| ------ | ------------ | ------------ | ---------- | ------------ |
| 1.3.0 | SGLang | 13.0 | 580.xx+ | - |
| 1.3.0 | TensorRT-LLM | 13.1 | 580.xx+ | - |
| 1.3.0 | vLLM | 13.0 | 580.xx+ | - |
| 1.2.1 | SGLang | 12.9 | 575.xx+ | - |
| 1.2.1 | SGLang | 13.0 | 580.xx+ | - |
| 1.2.1 | TensorRT-LLM | 13.1 | 580.xx+ | - |
| 1.2.1 | vLLM | 12.9 | 575.xx+ | - |
| 1.2.1 | vLLM | 13.0 | 580.xx+ | - |
| 1.2.0 | SGLang | 12.9 | 575.xx+ | - |
| 1.2.0 | SGLang | 13.0 | 580.xx+ | - |
| 1.2.0 | TensorRT-LLM | 13.1 | 580.xx+ | - |
| 1.2.0 | vLLM | 12.9 | 575.xx+ | - |
| 1.2.0 | vLLM | 13.0 | 580.xx+ | - |
| 1.1.1 | SGLang | 12.9 | 575.xx+ | - |
| 1.1.1 | SGLang | 13.0 | 580.xx+ | - |
| 1.1.1 | TensorRT-LLM | 13.1 | 580.xx+ | - |
| 1.1.1 | vLLM | 12.9 | 575.xx+ | - |
| 1.1.1 | vLLM | 13.0 | 580.xx+ | - |
| 1.1.0 | SGLang | 12.9 | 575.xx+ | - |
| 1.1.0 | SGLang | 13.0 | 580.xx+ | - |
| 1.1.0 | TensorRT-LLM | 13.1 | 580.xx+ | - |
| 1.1.0 | vLLM | 12.9 | 575.xx+ | - |
| 1.1.0 | vLLM | 13.0 | 580.xx+ | - |
| 1.0.2 | SGLang | 12.9 | 575.xx+ | - |
| 1.0.2 | SGLang | 13.0 | 580.xx+ | - |
| 1.0.2 | TensorRT-LLM | 13.1 | 580.xx+ | - |
| 1.0.2 | vLLM | 12.9 | 575.xx+ | - |
| 1.0.2 | vLLM | 13.0 | 580.xx+ | - |
| 1.0.1 | SGLang | 12.9 | 575.xx+ | - |
| 1.0.1 | SGLang | 13.0 | 580.xx+ | - |
| 1.0.1 | TensorRT-LLM | 13.1 | 580.xx+ | - |
| 1.0.1 | vLLM | 12.9 | 575.xx+ | - |
| 1.0.1 | vLLM | 13.0 | 580.xx+ | - |
| 1.0.0 | SGLang | 12.9 | 575.xx+ | - |
| 1.0.0 | SGLang | 13.0 | 580.xx+ | - |
| 1.0.0 | TensorRT-LLM | 13.1 | 580.xx+ | - |
| 1.0.0 | vLLM | 12.9 | 575.xx+ | - |
| 1.0.0 | vLLM | 13.0 | 580.xx+ | - |
| 0.9.1 | SGLang | 12.9 | 575.xx+ | - |
| 0.9.1 | TensorRT-LLM | 13.0 | 580.xx+ | - |
| 0.9.1 | vLLM | 12.9 | 575.xx+ | - |
| 0.9.0 | SGLang | 12.9 | 575.xx+ | - |
| 0.9.0 | TensorRT-LLM | 13.0 | 580.xx+ | - |
| 0.9.0 | vLLM | 12.9 | 575.xx+ | - |
| 0.8.1 | SGLang | 12.9 | 575.xx+ | - |
| 0.8.1 | SGLang | 13.0 | 580.xx+ | Experimental |
| 0.8.1 | TensorRT-LLM | 13.0 | 580.xx+ | - |
| 0.8.1 | vLLM | 12.9 | 575.xx+ | - |
| 0.8.1 | vLLM | 13.0 | 580.xx+ | Experimental |
| 0.8.0 | SGLang | 12.9 | 575.xx+ | - |
| 0.8.0 | SGLang | 13.0 | 580.xx+ | Experimental |
| 0.8.0 | TensorRT-LLM | 13.0 | 580.xx+ | - |
| 0.8.0 | vLLM | 12.9 | 575.xx+ | - |
| 0.8.0 | vLLM | 13.0 | 580.xx+ | Experimental |
| 0.7.1 | SGLang | 12.8 | 570.xx+ | - |
| 0.7.1 | TensorRT-LLM | 13.0 | 580.xx+ | - |
| 0.7.1 | vLLM | 12.9 | 575.xx+ | - |
| 0.7.0 | SGLang | 12.9 | 575.xx+ | - |
| 0.7.0 | TensorRT-LLM | 13.0 | 580.xx+ | - |
| 0.7.0 | vLLM | 12.8 | 570.xx+ | - |
* Patch versions (e.g. v0.8.1.post1, v0.7.0.post1) have the same CUDA support as their base version.
* Early access v1.1.0-dev.\* images follow the same CUDA matrix as v1.0.2. The v1.2.0-deepseek-v4-dev.3 vLLM container is CUDA 13.0 multi-arch; the SGLang containers split by arch (CUDA 12.9 on amd64, CUDA 13.0 on arm64).
* Experimental CUDA 13 images are not published for all versions.
**Feature support by backend (v1.3.0)**
| Feature | SGLang | TensorRT-LLM | vLLM |
| --------------------- | ---------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------- |
| Disaggregated Serving | Supported | Supported | Supported (Prefill/decode separation with NIXL KV transfer) |
| KV-Aware Routing | Supported | Supported | Supported |
| SLA-Based Planner | Supported | Supported | Supported |
| KV Block Manager | Experimental (Work in progress across all combinations) | Supported | Supported |
| Multimodal (Image) | Supported (Not compatible with KV-aware routing. Disagg patterns: EPD, E/PD, E/P/D (not traditional EP/D)) | Supported (Image URLs + pre-computed embeddings. Disagg: EP/D + E/P/D. KV-aware routing via dedicated MM Router Worker (requires KV event publishing)) | Supported (With KV-aware routing, image-aware routing on documented paths) |
| Multimodal (Video) | Supported | Not supported | Supported (Video input with frame sampling) |
| Multimodal (Audio) | Not supported | Not supported | Experimental (Qwen2-Audio, experimental) |
| Request Migration | Supported | Supported (Work in progress with multimodal) | Supported |
| Request Cancellation | Experimental (Remote-prefill-phase cancellation not supported in disaggregated mode) | Supported with caveat (Engine temporarily not notified of cancellations — resources for cancelled requests are not freed (known issue)) | Supported |
| LoRA | Not supported | Not supported | Supported (Dynamic load/unload; KV-aware routing supports adapter affinity) |
| Tool Calling | Supported | Supported | Supported |
| Speculative Decoding | Experimental (Code hooks exist; no examples or docs yet) | Supported | Supported (Eagle3) |
| Dynamo Snapshot | Supported | Not supported | Supported |
**Platform support**
* GPU architectures: Blackwell, Hopper, Ada Lovelace, Ampere
* OS: Ubuntu 24.04 (x86\_64, ARM64) — Supported
* OS: Ubuntu 22.04 (x86\_64) — Supported
* OS: CentOS Stream 9 (x86\_64) — Experimental
* CSP: AWS — Amazon Linux 2023 (x86\_64) — Supported
* CPU architectures: x86\_64, ARM64 (Ubuntu 24.04 only)
* Wheels: Wheels are built in a manylinux\_2\_28-compatible environment and validated on CentOS Stream 9 and Ubuntu 22.04/24.04. Other Linux distributions are expected to work but are not officially verified.