Support Matrix for NIMs#

This page lists the models, deployment profiles, and verified hardware SKUs supported for Dynamo mode in NIM LLM 3.0.

Only models with a published Dynamo recipe appear here. A model that is not listed may still be supported in standard NIM mode; refer to the support matrix for the release you are running.

Use the following sections to identify the supported deployment profiles for each model. Profile strings follow a naming convention described in Model Profiles and Selection.

Use the table below to filter profiles by GPU, tensor parallelism (TP), precision, LoRA support, and model name. Each row is one supported profile; details also appear in the per-model sections that follow.

These rows cover each model’s full profile set. For the subset validated with a published Dynamo recipe, refer to Dynamo Mode Support.

Note

The GB300-WS GPU is more commonly known as DGX Station.

LoRA only

Model	TP	Precision	LoRA
nemotron-3-super-120b-a12b	1	BF16	No
nemotron-3-super-120b-a12b	2	BF16	No
nemotron-3-super-120b-a12b	2	BF16	Yes
nemotron-3-super-120b-a12b	4	BF16	No
nemotron-3-super-120b-a12b	4	BF16	Yes
nemotron-3-super-120b-a12b	8	BF16	No
nemotron-3-super-120b-a12b	8	BF16	Yes
nemotron-3-super-120b-a12b	1	FP8	No
nemotron-3-super-120b-a12b	2	FP8	No
nemotron-3-super-120b-a12b	4	FP8	No
nemotron-3-super-120b-a12b	8	FP8	No
nemotron-3-super-120b-a12b	1	NVFP4	No
nemotron-3-super-120b-a12b	1	NVFP4	Yes
nemotron-3-super-120b-a12b	2	NVFP4	No
nemotron-3-super-120b-a12b	2	NVFP4	Yes
nemotron-3-super-120b-a12b	4	NVFP4	No
nemotron-3-super-120b-a12b	8	NVFP4	No
nemotron-3-ultra-550b-a55b	8	BF16	No
nemotron-3-ultra-550b-a55b	8	BF16	Yes
nemotron-3-ultra-550b-a55b	2	NVFP4	No
nemotron-3-ultra-550b-a55b	4	NVFP4	No
nemotron-3-ultra-550b-a55b	8	NVFP4	No
No matching profiles. This configuration may still be deployable with memory tuning; refer to the memory troubleshooting guide for details.

nemotron-3-super-120b-a12b#

The following table lists the supported profile configurations for nvidia/nemotron-3-super-120b-a12b:

Precision	TP1	TP2	TP4	TP8
BF16	`vllm-bf16-tp1-pp1`	`vllm-bf16-tp2-pp1`	`vllm-bf16-tp4-pp1`	`vllm-bf16-tp8-pp1`
BF16 + LoRA	–	`vllm-bf16-tp2-pp1-lora`	`vllm-bf16-tp4-pp1-lora`	`vllm-bf16-tp8-pp1-lora`
FP8	`vllm-fp8-tp1-pp1`	`vllm-fp8-tp2-pp1`	`vllm-fp8-tp4-pp1`	`vllm-fp8-tp8-pp1`
NVFP4	`vllm-nvfp4-tp1-pp1`	`vllm-nvfp4-tp2-pp1`	`vllm-nvfp4-tp4-pp1`	`vllm-nvfp4-tp8-pp1`
NVFP4 + LoRA	`vllm-nvfp4-tp1-pp1-lora`	`vllm-nvfp4-tp2-pp1-lora`	–	–

Note

This is a large model. Lower-TP profiles require substantially more GPU memory per device, so some verified GPUs support only TP4 or TP8 profiles.

nemotron-3-ultra-550b-a55b#

The following table lists the supported profile configurations for nvidia/nemotron-3-ultra-550b-a55b:

Precision	TP1	TP2	TP4	TP8
BF16	–	–	–	`vllm-bf16-tp8-pp1`
BF16 + LoRA	–	–	–	`vllm-bf16-tp8-pp1-lora`
NVFP4	–	`vllm-nvfp4-tp2-pp1`	`vllm-nvfp4-tp4-pp1`	`vllm-nvfp4-tp8-pp1`

Note

The minimum deployment is two GPUs, using vllm-nvfp4-tp2-pp1 on high-memory Blackwell SKUs (NVIDIA-B300-SXM6-AC and NVIDIA-GB300); the other profiles use four or eight GPUs. H200 validation covers one NVFP4 TP4 profile and is less extensive than the Blackwell validation.

FP8 profiles were not released for this NIM because NVFP4 profiles work on all tested Hopper-architecture GPUs (for example, H100 and H200) and deliver better performance than FP8 profiles.

Dynamo Mode Support#

Dynamo mode is generally available in NIM LLM 3.0. It runs NIM LLM on NVIDIA Dynamo, a distributed inference runtime. A Dynamo deployment separates the frontend, which exposes the OpenAI-compatible API and performs key-value (KV) cache-aware routing, from one or more workers that run the model.

The support scope for Dynamo mode is narrower than the support scope for standard NIM. The table below is self-contained: it is not reflected in the filter controls earlier on this page, and a GPU or profile that is verified for a standard NIM model section is not necessarily verified for Dynamo mode.

The release images, listed at the end of this section, include the NVIDIA Dynamo runtime. Keep the image, model profile, and worker arguments pinned by each published recipe together; substituting one independently is not a validated configuration. SGLang images are not supported for Dynamo mode.

For deployment shapes, orchestration, and component versions, refer to Dynamo Deployment and Dynamo Graph Deployment on Kubernetes. For the host requirements that apply to Dynamo mode, refer to Prerequisites.

GPUs and Profiles with Published Dynamo Recipes#

Download a model-specific deployment recipe from the model’s collection in the NGC Catalog. The following table lists the published GPU, precision, profile, and parallelism configurations for this release:

GPU SKUs and profiles with published Dynamo recipes#
Model	GPU	Precision	Profile	Tensor Parallelism	GPUs per Worker	Workloads
Nemotron-3-Super-120B-A12B	`NVIDIA-B200`	NVFP4	`vllm-nvfp4-tp4-pp1-24.0`	4	4	Chat, agentic
Nemotron-3-Super-120B-A12B	`NVIDIA-H200`	FP8	`vllm-fp8-tp4-pp1-35.0`	4	4	Chat, agentic
Nemotron-3-Ultra-550B-A55B	`NVIDIA-B200`	NVFP4	`vllm-nvfp4-tp4-pp1-90.0`	4	4	Chat, agentic
Nemotron-3-Ultra-550B-A55B	`NVIDIA-H200`	NVFP4	`vllm-nvfp4-tp8-pp1-49.0`	8	8	Chat, agentic

Each published recipe defines a validated image, profile, and configuration combination. Do not replace the recipe image independently. A model or GPU that does not appear in the table has no published NIM LLM 3.0 Dynamo recipe in this release.

All listed recipes use the vLLM backend framework and the per-worker GPU count shown in the table. Expert parallelism is enabled in every listed recipe, pipeline parallelism is 1, and data parallelism is not set. Replica counts and other deployment settings are defined by each recipe; refer to Dynamo Graph Deployment on Kubernetes.

The release recipes use the following images:

Nemotron-3-Super-120B-A12B: nvcr.io/nim/nvidia/nemotron-3-super-120b-a12b:3.0.0
Nemotron-3-Ultra-550B-A55B: nvcr.io/nim/nvidia/model-free-nim:3.0.0