Compatibility | NVIDIA Dynamo Documentation

Compatibility by version

Dynamo v1.3.1Patch

Dynamo version

Released Aug 5, 2026 · Release notes · UCX 1.20.x

SGLang0.5.14

NIXL1.3.2

CUDA 13.0Driver 580.xx+

TensorRT-LLM1.3.0rc19

NIXL1.0.1

CUDA 13.1Driver 580.xx+

vLLM0.23.0

NIXL1.1.0

CUDA 13.0Driver 580.xx+

GPU

BlackwellHopperAda LovelaceAmpere

Ubuntu 24.04Ubuntu 22.04CentOS Stream 9 · experimental

Arch

x86_64ARM64 (Ubuntu 24.04 only)

Patch release. Fixes disaggregated SGLang serving over AWS EFA on GB200: the SGLang EFA runtime moves to NIXL 1.3.2 and all three EFA images to EFA Installer 1.49.0. Backend pins are otherwise unchanged from v1.3.0.

Backend versions listed are the versions tested and supported for the selected release. TensorRT-LLM does not support Python 3.11.

For extended driver compatibility beyond the listed minimums, including forward compatibility and cuda-compat packages, see the CUDA Compatibility documentation.

See Release Artifacts for the full artifact inventory — container images, wheels, Helm charts, and crates — and Model Early Access Builds for per-model early access container builds.

Release Support Matrix

The card above covers one release at a time. Expand the matrix below to compare CUDA toolkit and minimum driver requirements across releases, per backend.

CUDA toolkit and minimum driver by release

Every released line — stable releases and their patches, grouped by minor line, newest first. Platform previews and model-specific builds are not listed individually; those with a documented toolkit requirement appear in the notes below, and Releases (machine-readable) has the full release inventory.

1.3.x

Dynamo	Backend	CUDA Toolkit	Min Driver	Note
1.3.1	SGLang	13.0	580.xx+	-
1.3.1	TensorRT-LLM	13.1	580.xx+	-
1.3.1	vLLM	13.0	580.xx+	-
1.3.0	SGLang	13.0	580.xx+	-
1.3.0	TensorRT-LLM	13.1	580.xx+	-
1.3.0	vLLM	13.0	580.xx+	-

1.2.x

Dynamo	Backend	CUDA Toolkit	Min Driver	Note
1.2.1	SGLang	12.9	575.xx+	-
1.2.1	SGLang	13.0	580.xx+	-
1.2.1	TensorRT-LLM	13.1	580.xx+	-
1.2.1	vLLM	12.9	575.xx+	-
1.2.1	vLLM	13.0	580.xx+	-
1.2.0	SGLang	12.9	575.xx+	-
1.2.0	SGLang	13.0	580.xx+	-
1.2.0	TensorRT-LLM	13.1	580.xx+	-
1.2.0	vLLM	12.9	575.xx+	-
1.2.0	vLLM	13.0	580.xx+	-

1.1.x

Dynamo	Backend	CUDA Toolkit	Min Driver	Note
1.1.1	SGLang	12.9	575.xx+	-
1.1.1	SGLang	13.0	580.xx+	-
1.1.1	TensorRT-LLM	13.1	580.xx+	-
1.1.1	vLLM	12.9	575.xx+	-
1.1.1	vLLM	13.0	580.xx+	-
1.1.0	SGLang	12.9	575.xx+	-
1.1.0	SGLang	13.0	580.xx+	-
1.1.0	TensorRT-LLM	13.1	580.xx+	-
1.1.0	vLLM	12.9	575.xx+	-
1.1.0	vLLM	13.0	580.xx+	-

1.0.x

Dynamo	Backend	CUDA Toolkit	Min Driver	Note
1.0.2	SGLang	12.9	575.xx+	-
1.0.2	SGLang	13.0	580.xx+	-
1.0.2	TensorRT-LLM	13.1	580.xx+	-
1.0.2	vLLM	12.9	575.xx+	-
1.0.2	vLLM	13.0	580.xx+	-
1.0.1	SGLang	12.9	575.xx+	-
1.0.1	SGLang	13.0	580.xx+	-
1.0.1	TensorRT-LLM	13.1	580.xx+	-
1.0.1	vLLM	12.9	575.xx+	-
1.0.1	vLLM	13.0	580.xx+	-
1.0.0	SGLang	12.9	575.xx+	-
1.0.0	SGLang	13.0	580.xx+	-
1.0.0	TensorRT-LLM	13.1	580.xx+	-
1.0.0	vLLM	12.9	575.xx+	-
1.0.0	vLLM	13.0	580.xx+	-

0.9.x

Dynamo	Backend	CUDA Toolkit	Min Driver	Note
0.9.1	SGLang	12.9	575.xx+	-
0.9.1	TensorRT-LLM	13.0	580.xx+	-
0.9.1	vLLM	12.9	575.xx+	-
0.9.0	SGLang	12.9	575.xx+	-
0.9.0	TensorRT-LLM	13.0	580.xx+	-
0.9.0	vLLM	12.9	575.xx+	-

0.8.x

Dynamo	Backend	CUDA Toolkit	Min Driver	Note
0.8.1	SGLang	12.9	575.xx+	-
0.8.1	SGLang	13.0	580.xx+	Experimental
0.8.1	TensorRT-LLM	13.0	580.xx+	-
0.8.1	vLLM	12.9	575.xx+	-
0.8.1	vLLM	13.0	580.xx+	Experimental
0.8.0	SGLang	12.9	575.xx+	-
0.8.0	SGLang	13.0	580.xx+	Experimental
0.8.0	TensorRT-LLM	13.0	580.xx+	-
0.8.0	vLLM	12.9	575.xx+	-
0.8.0	vLLM	13.0	580.xx+	Experimental

0.7.x

Dynamo	Backend	CUDA Toolkit	Min Driver	Note
0.7.1	SGLang	12.8	570.xx+	-
0.7.1	TensorRT-LLM	13.0	580.xx+	-
0.7.1	vLLM	12.9	575.xx+	-
0.7.0	SGLang	12.9	575.xx+	-
0.7.0	TensorRT-LLM	13.0	580.xx+	-
0.7.0	vLLM	12.8	570.xx+	-

Patch versions (e.g. v0.8.1.post1, v0.7.0.post1) have the same CUDA support as their base version.
Early access v1.1.0-dev.* images follow the same CUDA matrix as v1.0.2. The v1.2.0-deepseek-v4-dev.3 vLLM container is CUDA 13.0 multi-arch; the SGLang containers split by arch (CUDA 12.9 on amd64, CUDA 13.0 on arm64).
Experimental CUDA 13 images are not published for all versions.

Platform Notes

Dynamo ships multi-arch (x86_64 + ARM64) container images. Wheels are built in a manylinux_2_28-compatible environment and validated on CentOS Stream 9 and Ubuntu 22.04/24.04; other Linux distributions are expected to work but are not officially verified.

Cloud Service Providers

Amazon Linux 2023 (AWS) · x86_64 · Supported

AL2023 TensorRT-LLM limitation: there is a known issue with the TensorRT-LLM framework when running the AL2023 container locally with docker run --network host ... due to a bug in mpi4py. Replace the --network host flag with precise networking configuration by mapping only the necessary ports (4222 for NATS, 2379/2380 for etcd, 8000 for the frontend).

Feature Support

Feature support by backend

SupportedCaveatExperimentalNot supported

SGLang9 / 15

TRT-LLM9 / 15

vLLM14 / 15

Disaggregated Serving

KV-Aware Routing

SLA-Based Planner

KV Block Manager

Multimodal (Image)

⁴

⁵

Multimodal (Video)

—

⁶

Multimodal (Audio)

—

⁷

Request Migration

⁸

Request Cancellation

⁹

¹⁰

LoRA

—

¹¹

Tool Calling

Speculative Decoding

¹²

¹³

GPU Memory Service

¹⁴

¹⁵

¹⁶

Shadow Engine Failover

¹⁷

¹⁸

¹⁹

Dynamo Snapshot

²⁰

²¹

²²

Disaggregated Serving · vLLM: Prefill/decode separation with NIXL KV transfer
KV Block Manager · SGLang: Work in progress across all combinations
Multimodal (Image) · SGLang: Not compatible with KV-aware routing. Disagg patterns: EPD, E/PD, E/P/D (not traditional EP/D)
Multimodal (Image) · TRT-LLM: Image URLs + pre-computed embeddings. Disagg: EP/D + E/P/D. KV-aware routing via dedicated MM Router Worker (requires KV event publishing)
Multimodal (Image) · vLLM: With KV-aware routing, image-aware routing on documented paths
Multimodal (Video) · vLLM: Video input with frame sampling
Multimodal (Audio) · vLLM: Qwen2-Audio, experimental
Request Migration · TRT-LLM: Work in progress with multimodal
Request Cancellation · SGLang: Remote-prefill-phase cancellation not supported in disaggregated mode
Request Cancellation · TRT-LLM: Engine temporarily not notified of cancellations — resources for cancelled requests are not freed (known issue)
LoRA · vLLM: Dynamic load/unload; KV-aware routing supports adapter affinity
Speculative Decoding · SGLang: Code hooks exist; no examples or docs yet
Speculative Decoding · vLLM: Eagle3
GPU Memory Service · SGLang: Weights and KV; upstream integration remains in progress
GPU Memory Service · TRT-LLM: Weights only; multinode and upstream integration remain in progress
GPU Memory Service · vLLM: Weights and KV; upstream integration remains in progress
Shadow Engine Failover · SGLang: No KV-cache reuse or hardware fault tolerance
Shadow Engine Failover · TRT-LLM: No KV-cache reuse or hardware fault tolerance
Shadow Engine Failover · vLLM: Software-process failover only; no KV-cache reuse or hardware fault tolerance
Dynamo Snapshot · SGLang: Single-GPU supported; multi-GPU and multinode remain in progress
Dynamo Snapshot · TRT-LLM: Single-GPU aggregated text-worker path only
Dynamo Snapshot · vLLM: Single-GPU supported; multi-GPU is highly experimental and multinode remains in progress

Superscripts reference the numbered notes above; full per-backend detail follows.

Per-Backend Detail

vLLM

SGLang

TensorRT-LLM

vLLM offers the broadest feature coverage in Dynamo, with full support for disaggregated serving, KV-aware routing, KV block management, LoRA adapters, and multimodal inference including video and audio.

Source: docs/backends/vllm/README.md

Feature	Supported?	Notes
Disaggregated Serving	✓	Prefill/decode separation with NIXL KV transfer
KV-Aware Routing	✓
SLA-Based Planner	✓
KV Block Manager	✓
Multimodal	✓	Image + video; audio experimental (Qwen2-Audio). With KV-aware routing, image-aware routing on documented paths (Source)
Request Migration	✓
Request Cancellation	✓
LoRA	✓	Dynamic load/unload; KV-aware routing supports adapter affinity
Tool Calling	✓
Speculative Decoding	✓	Eagle3 (Source)
GPU Memory Service	✓	Weights and KV; upstream integration remains in progress
Shadow Engine Failover	!	Software-process failover only; no KV-cache reuse or hardware fault tolerance
Dynamo Snapshot	!	Single-GPU supported; multi-GPU is highly experimental and multinode remains in progress

Feature Interactions

Pairwise feature-by-feature compatibility within each backend. Each cell reports whether the row feature works together with the column feature. A — marks the diagonal or a combination that does not apply; blank cells are the mirror of the populated lower triangle.

Legend: ✓ Supported · WIP Work in Progress / Experimental / Limited

vLLM Feature Interactions

Feature	Disaggregated Serving	KV-Aware Routing	SLA-Based Planner	KV Block Manager	Multimodal	Request Migration	Request Cancellation	LoRA	Tool Calling	Speculative Decoding
Disaggregated Serving	—
KV-Aware Routing	✅	—
SLA-Based Planner	✅	✅	—
KV Block Manager	✅	✅	✅	—
Multimodal	✅	✅¹	—	✅	—
Request Migration	✅	✅	✅	✅	✅	—
Request Cancellation	✅	✅	✅	✅	✅	✅	—
LoRA	✅	✅²	—	✅	—	✅	✅	—
Tool Calling	✅	✅	✅	✅	✅	✅	✅	✅	—
Speculative Decoding	✅	✅	—	✅	—	✅	✅	—	✅	—

Notes:

Multimodal + KV-Aware Routing: Image-aware KV routing is supported in the documented vLLM paths. The default Rust frontend path supports model families handled by llm-multimodal; the Python chat-processor path delegates to vLLM’s multimodal processor. (Source)

KV-Aware LoRA Routing: vLLM supports routing requests based on LoRA adapter affinity.

Audio Support: vLLM supports audio models like Qwen2-Audio (experimental). (Source)

Video Support: vLLM supports video input with frame sampling. (Source)

Speculative Decoding: Eagle3 support documented. (Source)

SGLang Feature Interactions

Feature	Disaggregated Serving	KV-Aware Routing	SLA-Based Planner	KV Block Manager	Multimodal	Request Migration	Request Cancellation	LoRA	Tool Calling	Speculative Decoding
Disaggregated Serving	—
KV-Aware Routing	✅	—
SLA-Based Planner	✅	✅	—
KV Block Manager	🚧	🚧	🚧	—
Multimodal	✅²	¹	—	🚧	—
Request Migration	✅	✅	✅	🚧	✅	—
Request Cancellation	🚧³	✅	✅	🚧	🚧	✅	—
LoRA				🚧				—
Tool Calling	✅	✅	✅	🚧	✅	✅	✅		—
Speculative Decoding	🚧	🚧	—	🚧	—	🚧	—		🚧	—

Notes:

Multimodal + KV-Aware Routing: Not supported. (Source)

Multimodal Patterns: Supports simple Aggregated EPD, E/PD, and E/P/D patterns. Traditional Disagg EP/D is not supported. (Source)

Request Cancellation: Cancellation during the remote prefill phase is not supported in disaggregated mode. (Source)

Speculative Decoding: Code hooks exist (spec_decode_stats in publisher), but no examples or documentation yet.

TensorRT-LLM Feature Interactions

Feature	Disaggregated Serving	KV-Aware Routing	SLA-Based Planner	KV Block Manager	Multimodal	Request Migration	Request Cancellation	LoRA	Tool Calling	Speculative Decoding
Disaggregated Serving	—
KV-Aware Routing	✅	—
SLA-Based Planner	✅	✅	—
KV Block Manager	✅	✅	✅	—
Multimodal	✅¹	✅²	—	✅	—
Request Migration	✅	✅	✅	✅	🚧	—
Request Cancellation	✅³	✅³	✅³	✅³	✅³	✅³	—
LoRA								—
Tool Calling	✅	✅	✅	✅	✅	✅	✅		—
Speculative Decoding	✅	✅	—	✅	—	✅	✅		✅	—

Notes:

Multimodal Disaggregation: Supports EP/D (Traditional) and E/P/D (Full Disaggregation) image flows, including image URLs and pre-computed embeddings. (Source)

Multimodal + KV-Aware Routing: The native Rust frontend routes supported models using image-aware KV overlap. TRT-LLM workers must publish KV events with block reuse enabled. (Source)

Request Cancellation: Due to known issues, the TensorRT-LLM engine is temporarily not notified of request cancellations, meaning allocated resources for cancelled requests are not freed.