Support Matrix for NIMs#

This page lists supported models, deployment profiles, and verified hardware SKUs for NIM LLM.

Use the following sections to identify the supported deployment profiles for each model. Profile strings follow a naming convention described in Model Profiles and Selection.

NVIDIA publishes inference microservices (NIMs) under distinct NIM offerings. Models on this page are grouped in separate sections based on NIM offering: NIM Certified and NIM. For more information, refer to NIM Offerings.

Use the table below to filter NIM profiles by GPU, tensor parallelism (TP), precision, certification status, and model name. Each row is one supported profile; details also appear in the per-model sections that follow.

LoRA only Certified only

Model	TP	Precision	LoRA
gpt-oss-120b	1	MXFP4	No
gpt-oss-120b	2	MXFP4	No
gpt-oss-120b	4	MXFP4	No
gpt-oss-120b	8	MXFP4	No
gpt-oss-120b	1	MXFP4	Yes
gpt-oss-120b	2	MXFP4	Yes
gpt-oss-120b	4	MXFP4	Yes
gpt-oss-120b	8	MXFP4	Yes
gpt-oss-20b	1	MXFP4	No
gpt-oss-20b	2	MXFP4	No
gpt-oss-20b	4	MXFP4	No
gpt-oss-20b	8	MXFP4	No
gpt-oss-20b	1	MXFP4	Yes
gpt-oss-20b	2	MXFP4	Yes
gpt-oss-20b	4	MXFP4	Yes
gpt-oss-20b	8	MXFP4	Yes
llama-3.1-70b-instruct	1	BF16	No
llama-3.1-70b-instruct	2	BF16	No
llama-3.1-70b-instruct	4	BF16	No
llama-3.1-70b-instruct	8	BF16	No
llama-3.1-70b-instruct	1	BF16	Yes
llama-3.1-70b-instruct	2	BF16	Yes
llama-3.1-70b-instruct	4	BF16	Yes
llama-3.1-70b-instruct	8	BF16	Yes
llama-3.1-70b-instruct	1	FP8	No
llama-3.1-70b-instruct	2	FP8	No
llama-3.1-70b-instruct	4	FP8	No
llama-3.1-70b-instruct	8	FP8	No
llama-3.1-70b-instruct	1	FP8	Yes
llama-3.1-70b-instruct	2	FP8	Yes
llama-3.1-70b-instruct	4	FP8	Yes
llama-3.1-70b-instruct	8	FP8	Yes
llama-3.1-70b-instruct	1	NVFP4	No
llama-3.1-70b-instruct	2	NVFP4	No
llama-3.1-70b-instruct	4	NVFP4	No
llama-3.1-70b-instruct	8	NVFP4	No
llama-3.1-70b-instruct	1	NVFP4	Yes
llama-3.1-70b-instruct	2	NVFP4	Yes
llama-3.1-70b-instruct	4	NVFP4	Yes
llama-3.1-70b-instruct	8	NVFP4	Yes
llama-3.1-8b-instruct	1	BF16	No
llama-3.1-8b-instruct	1	BF16	Yes
llama-3.1-8b-instruct	1	FP8	No
llama-3.1-8b-instruct	1	FP8	Yes
llama-3.1-8b-instruct	1	NVFP4	No
llama-3.1-8b-instruct	1	NVFP4	Yes
llama-3.3-70b-instruct	1	BF16	No
llama-3.3-70b-instruct	2	BF16	No
llama-3.3-70b-instruct	4	BF16	No
llama-3.3-70b-instruct	8	BF16	No
llama-3.3-70b-instruct	1	BF16	Yes
llama-3.3-70b-instruct	2	BF16	Yes
llama-3.3-70b-instruct	4	BF16	Yes
llama-3.3-70b-instruct	8	BF16	Yes
llama-3.3-70b-instruct	1	FP8	No
llama-3.3-70b-instruct	2	FP8	No
llama-3.3-70b-instruct	4	FP8	No
llama-3.3-70b-instruct	8	FP8	No
llama-3.3-70b-instruct	1	FP8	Yes
llama-3.3-70b-instruct	2	FP8	Yes
llama-3.3-70b-instruct	4	FP8	Yes
llama-3.3-70b-instruct	8	FP8	Yes
llama-3.3-70b-instruct	1	NVFP4	No
llama-3.3-70b-instruct	2	NVFP4	No
llama-3.3-70b-instruct	4	NVFP4	No
llama-3.3-70b-instruct	8	NVFP4	No
llama-3.3-70b-instruct	1	NVFP4	Yes
llama-3.3-70b-instruct	2	NVFP4	Yes
llama-3.3-70b-instruct	4	NVFP4	Yes
llama-3.3-70b-instruct	8	NVFP4	Yes
llama-3.3-nemotron-super-49b-v1.5	1	BF16	No
llama-3.3-nemotron-super-49b-v1.5	2	BF16	No
llama-3.3-nemotron-super-49b-v1.5	4	BF16	No
llama-3.3-nemotron-super-49b-v1.5	8	BF16	No
llama-3.3-nemotron-super-49b-v1.5	1	BF16	Yes
llama-3.3-nemotron-super-49b-v1.5	2	BF16	Yes
llama-3.3-nemotron-super-49b-v1.5	4	BF16	Yes
llama-3.3-nemotron-super-49b-v1.5	8	BF16	Yes
llama-3.3-nemotron-super-49b-v1.5	1	FP8	No
llama-3.3-nemotron-super-49b-v1.5	2	FP8	No
llama-3.3-nemotron-super-49b-v1.5	4	FP8	No
llama-3.3-nemotron-super-49b-v1.5	8	FP8	No
llama-3.3-nemotron-super-49b-v1.5	1	FP8	Yes
llama-3.3-nemotron-super-49b-v1.5	2	FP8	Yes
llama-3.3-nemotron-super-49b-v1.5	4	FP8	Yes
llama-3.3-nemotron-super-49b-v1.5	8	FP8	Yes
llama-3.3-nemotron-super-49b-v1.5	1	NVFP4	No
llama-3.3-nemotron-super-49b-v1.5	2	NVFP4	No
llama-3.3-nemotron-super-49b-v1.5	4	NVFP4	No
llama-3.3-nemotron-super-49b-v1.5	8	NVFP4	No
llama-3.3-nemotron-super-49b-v1.5	1	NVFP4	Yes
llama-3.3-nemotron-super-49b-v1.5	2	NVFP4	Yes
llama-3.3-nemotron-super-49b-v1.5	4	NVFP4	Yes
llama-3.3-nemotron-super-49b-v1.5	8	NVFP4	Yes
nemotron-3-nano	1	BF16	No
nemotron-3-nano	2	BF16	No
nemotron-3-nano	4	BF16	No
nemotron-3-nano	8	BF16	No
nemotron-3-nano	1	BF16	Yes
nemotron-3-nano	2	BF16	Yes
nemotron-3-nano	4	BF16	Yes
nemotron-3-nano	8	BF16	Yes
nemotron-3-nano	1	FP8	No
nemotron-3-nano	2	FP8	No
nemotron-3-nano	4	FP8	No
nemotron-3-nano	8	FP8	No
nemotron-3-nano	1	NVFP4	No
nemotron-3-nano	2	NVFP4	No
nemotron-3-nano	4	NVFP4	No
nemotron-3-nano	8	NVFP4	No
nemotron-3-nano	1	NVFP4	Yes
nemotron-3-super-120b-a12b	1	BF16	No
nemotron-3-super-120b-a12b	2	BF16	No
nemotron-3-super-120b-a12b	4	BF16	No
nemotron-3-super-120b-a12b	8	BF16	No
nemotron-3-super-120b-a12b	2	BF16	Yes
nemotron-3-super-120b-a12b	4	BF16	Yes
nemotron-3-super-120b-a12b	8	BF16	Yes
nemotron-3-super-120b-a12b	1	FP8	No
nemotron-3-super-120b-a12b	2	FP8	No
nemotron-3-super-120b-a12b	4	FP8	No
nemotron-3-super-120b-a12b	8	FP8	No
nemotron-3-super-120b-a12b	1	NVFP4	No
nemotron-3-super-120b-a12b	2	NVFP4	No
nemotron-3-super-120b-a12b	4	NVFP4	No
nemotron-3-super-120b-a12b	8	NVFP4	No
nemotron-3-super-120b-a12b	1	NVFP4	Yes
nemotron-3-super-120b-a12b	2	NVFP4	Yes
nemotron-3-ultra-550b-a55b	8	BF16	No
nemotron-3-ultra-550b-a55b	8	BF16	Yes
nemotron-3-ultra-550b-a55b	2	NVFP4	No
nemotron-3-ultra-550b-a55b	4	NVFP4	No
nemotron-3-ultra-550b-a55b	8	NVFP4	No
riva-translate-4b-instruct-v2	1	BF16	No
starcoder2-7b	1	BF16	No
starcoder2-7b	2	BF16	No
nemotron-content-safety-reasoning-4b-experimental	1	BF16	No
nemotron-content-safety-reasoning-4b-experimental	2	BF16	No
nemotron-content-safety-reasoning-4b-experimental	4	BF16	No
nemotron-content-safety-reasoning-4b-experimental	8	BF16	No
No matching profiles. This configuration may still be deployable with memory tuning; refer to the memory troubleshooting guide for details.
No matching profiles. This GPU is verified for Model-Free NIM deployment, and this configuration may also be deployable with memory tuning; refer to the memory troubleshooting guide for details.

NIM Certified#

NIM Certified is the enterprise production NIM offering. It requires NVIDIA AI Enterprise.

gpt-oss-120b#

The following table lists the supported profile configurations for openai/gpt-oss-120b:

Precision	TP1	TP2	TP4	TP8
MXFP4	`vllm-mxfp4-tp1-pp1`	`vllm-mxfp4-tp2-pp1`	`vllm-mxfp4-tp4-pp1`	`vllm-mxfp4-tp8-pp1`
MXFP4 + LoRA	`vllm-mxfp4-tp1-pp1-lora`	`vllm-mxfp4-tp2-pp1-lora`	`vllm-mxfp4-tp4-pp1-lora`	`vllm-mxfp4-tp8-pp1-lora`

gpt-oss-20b#

The following table lists the supported profile configurations for openai/gpt-oss-20b:

Precision	TP1	TP2	TP4	TP8
MXFP4	`vllm-mxfp4-tp1-pp1`	`vllm-mxfp4-tp2-pp1`	`vllm-mxfp4-tp4-pp1`	`vllm-mxfp4-tp8-pp1`
MXFP4 + LoRA	`vllm-mxfp4-tp1-pp1-lora`	`vllm-mxfp4-tp2-pp1-lora`	`vllm-mxfp4-tp4-pp1-lora`	`vllm-mxfp4-tp8-pp1-lora`

llama-3.1-70b-instruct#

The following table lists the supported profile configurations for meta/llama-3.1-70b-instruct:

Precision	TP1	TP2	TP4	TP8
BF16	`vllm-bf16-tp1-pp1`	`vllm-bf16-tp2-pp1`	`vllm-bf16-tp4-pp1`	`vllm-bf16-tp8-pp1`
BF16 + LoRA	`vllm-bf16-tp1-pp1-lora`	`vllm-bf16-tp2-pp1-lora`	`vllm-bf16-tp4-pp1-lora`	`vllm-bf16-tp8-pp1-lora`
FP8	`vllm-fp8-tp1-pp1`	`vllm-fp8-tp2-pp1`	`vllm-fp8-tp4-pp1`	`vllm-fp8-tp8-pp1`
FP8 + LoRA	`vllm-fp8-tp1-pp1-lora`	`vllm-fp8-tp2-pp1-lora`	`vllm-fp8-tp4-pp1-lora`	`vllm-fp8-tp8-pp1-lora`
NVFP4	`vllm-nvfp4-tp1-pp1`	`vllm-nvfp4-tp2-pp1`	`vllm-nvfp4-tp4-pp1`	`vllm-nvfp4-tp8-pp1`
NVFP4 + LoRA	`vllm-nvfp4-tp1-pp1-lora`	`vllm-nvfp4-tp2-pp1-lora`	`vllm-nvfp4-tp4-pp1-lora`	`vllm-nvfp4-tp8-pp1-lora`

llama-3.1-8b-instruct#

The following table lists the supported profile configurations for meta/llama-3.1-8b-instruct:

Precision	TP1
BF16	`vllm-bf16-tp1-pp1`
BF16 + LoRA	`vllm-bf16-tp1-pp1-lora`
FP8	`vllm-fp8-tp1-pp1`
FP8 + LoRA	`vllm-fp8-tp1-pp1-lora`
NVFP4	`vllm-nvfp4-tp1-pp1`
NVFP4 + LoRA	`vllm-nvfp4-tp1-pp1-lora`

llama-3.3-70b-instruct#

The following table lists the supported profile configurations for meta/llama-3.3-70b-instruct:

Precision	TP1	TP2	TP4	TP8
BF16	`vllm-bf16-tp1-pp1`	`vllm-bf16-tp2-pp1`	`vllm-bf16-tp4-pp1`	`vllm-bf16-tp8-pp1`
BF16 + LoRA	`vllm-bf16-tp1-pp1-lora`	`vllm-bf16-tp2-pp1-lora`	`vllm-bf16-tp4-pp1-lora`	`vllm-bf16-tp8-pp1-lora`
FP8	`vllm-fp8-tp1-pp1`	`vllm-fp8-tp2-pp1`	`vllm-fp8-tp4-pp1`	`vllm-fp8-tp8-pp1`
FP8 + LoRA	`vllm-fp8-tp1-pp1-lora`	`vllm-fp8-tp2-pp1-lora`	`vllm-fp8-tp4-pp1-lora`	`vllm-fp8-tp8-pp1-lora`
NVFP4	`vllm-nvfp4-tp1-pp1`	`vllm-nvfp4-tp2-pp1`	`vllm-nvfp4-tp4-pp1`	`vllm-nvfp4-tp8-pp1`
NVFP4 + LoRA	`vllm-nvfp4-tp1-pp1-lora`	`vllm-nvfp4-tp2-pp1-lora`	`vllm-nvfp4-tp4-pp1-lora`	`vllm-nvfp4-tp8-pp1-lora`

llama-3.3-nemotron-super-49b-v1.5#

The following table lists the supported profile configurations for nvidia/llama-3.3-nemotron-super-49b-v1.5:

Precision	TP1	TP2	TP4	TP8
BF16	`vllm-bf16-tp1-pp1`	`vllm-bf16-tp2-pp1`	`vllm-bf16-tp4-pp1`	`vllm-bf16-tp8-pp1`
BF16 + LoRA	`vllm-bf16-tp1-pp1-lora`	`vllm-bf16-tp2-pp1-lora`	`vllm-bf16-tp4-pp1-lora`	`vllm-bf16-tp8-pp1-lora`
FP8	`vllm-fp8-tp1-pp1`	`vllm-fp8-tp2-pp1`	`vllm-fp8-tp4-pp1`	`vllm-fp8-tp8-pp1`
FP8 + LoRA	`vllm-fp8-tp1-pp1-lora`	`vllm-fp8-tp2-pp1-lora`	`vllm-fp8-tp4-pp1-lora`	`vllm-fp8-tp8-pp1-lora`
NVFP4	`vllm-nvfp4-tp1-pp1`	`vllm-nvfp4-tp2-pp1`	`vllm-nvfp4-tp4-pp1`	`vllm-nvfp4-tp8-pp1`
NVFP4 + LoRA	`vllm-nvfp4-tp1-pp1-lora`	`vllm-nvfp4-tp2-pp1-lora`	`vllm-nvfp4-tp4-pp1-lora`	`vllm-nvfp4-tp8-pp1-lora`

nemotron-3-nano#

The following table lists the supported profile configurations for nvidia/nemotron-3-nano:

Precision	TP1	TP2	TP4	TP8
BF16	`vllm-bf16-tp1-pp1`	`vllm-bf16-tp2-pp1`	`vllm-bf16-tp4-pp1`	`vllm-bf16-tp8-pp1`
BF16 + LoRA	`vllm-bf16-tp1-pp1-lora`	`vllm-bf16-tp2-pp1-lora`	`vllm-bf16-tp4-pp1-lora`	`vllm-bf16-tp8-pp1-lora`
FP8	`vllm-fp8-tp1-pp1`	`vllm-fp8-tp2-pp1`	`vllm-fp8-tp4-pp1`	`vllm-fp8-tp8-pp1`
NVFP4	`vllm-nvfp4-tp1-pp1`	`vllm-nvfp4-tp2-pp1`	`vllm-nvfp4-tp4-pp1`	`vllm-nvfp4-tp8-pp1`
NVFP4 + LoRA	`vllm-nvfp4-tp1-pp1-lora`	–	–	–

nemotron-3-super-120b-a12b#

The following table lists the supported profile configurations for nvidia/nemotron-3-super-120b-a12b:

Precision	TP1	TP2	TP4	TP8
BF16	`vllm-bf16-tp1-pp1`	`vllm-bf16-tp2-pp1`	`vllm-bf16-tp4-pp1`	`vllm-bf16-tp8-pp1`
BF16 + LoRA	–	`vllm-bf16-tp2-pp1-lora`	`vllm-bf16-tp4-pp1-lora`	`vllm-bf16-tp8-pp1-lora`
FP8	`vllm-fp8-tp1-pp1`	`vllm-fp8-tp2-pp1`	`vllm-fp8-tp4-pp1`	`vllm-fp8-tp8-pp1`
NVFP4	`vllm-nvfp4-tp1-pp1`	`vllm-nvfp4-tp2-pp1`	`vllm-nvfp4-tp4-pp1`	`vllm-nvfp4-tp8-pp1`
NVFP4 + LoRA	`vllm-nvfp4-tp1-pp1-lora`	`vllm-nvfp4-tp2-pp1-lora`	–	–

Note

This is a large model. Lower-TP profiles require substantially more GPU memory per device, so some verified GPUs support only TP4 or TP8 profiles.

nemotron-3-ultra-550b-a55b#

The following table lists the supported profile configurations for nvidia/nemotron-3-ultra-550b-a55b:

Precision	TP1	TP2	TP4	TP8
BF16	–	–	–	`vllm-bf16-tp8-pp1`
BF16 + LoRA	–	–	–	`vllm-bf16-tp8-pp1-lora`
NVFP4	–	`vllm-nvfp4-tp2-pp1`	`vllm-nvfp4-tp4-pp1`	`vllm-nvfp4-tp8-pp1`

Note

The minimum deployment is two GPUs, using vllm-nvfp4-tp2-pp1 on high-memory Blackwell SKUs (NVIDIA-B300-SXM6-AC and NVIDIA-GB300); the other profiles use four or eight GPUs. H200 validation covers one NVFP4 TP4 profile and is less extensive than the Blackwell validation.

riva-translate-4b-instruct-v2#

The following table lists the supported profile configuration for nvidia/riva-translate-4b-instruct-v2:

Precision	TP	PP	LoRA	Profile
BF16	1	1	No	`vllm-bf16-tp1-pp1`

starcoder2-7b#

The following table lists the supported profile configurations for bigcode/starcoder2-7b:

Precision	TP1	TP2
BF16	`vllm-bf16-tp1-pp1`	`vllm-bf16-tp2-pp1`

Model-Free NIM#

The following models are tested and validated for nvidia/model-free-nim:

gpt-oss-20b
apriel-nemotron
codestral

While not explicitly validated, the model-free NIM can be used with any model supported by the underlying backend (vLLM) version. Refer to Model-Free NIM for deployment details.

NIM#

NIM is the offering that delivers NIMs that are validated to be functional on a small set of NVIDIA GPUs. It is free to use and is not part of the NVIDIA AI Enterprise portfolio.

nemotron-content-safety-reasoning-4b-experimental#

The following table lists the supported profile configurations for nvidia/nemotron-content-safety-reasoning-4b-experimental:

Precision	TP1	TP2	TP4	TP8
BF16	`vllm-bf16-tp1-pp1`	`vllm-bf16-tp2-pp1`	`vllm-bf16-tp4-pp1`	`vllm-bf16-tp8-pp1`

For information on getting started with nemotron-content-safety-reasoning-4b-experimental, refer to version 2.0.3 of the documentation.

1.x NIM LLM Models#

For more information on version 1.x NIMs, refer to the 1.15 version of the NIM LLM Supported Models page.

Show 1.x models

Model (Hardware Requirements)	Organization/Model ID (Catalog Page)
DeepSeek-V3.1-Terminus	`deepseek-ai/deepseek-v3.1-terminus`
DeepSeek-V3.2-Exp	`deepseek-ai/deepseek-v32-exp-nim`
GLM-5	`zai-org/glm-5`
GLM-5.1	`zai-org/glm-51`
Llama-3.1-Nemotron-Nano-8B-Healthcare-Text2sql-v1.0	`nvidia/llama-3.1-nemotron-nano-8b-healthcare-text2sql-v1.0`
Llama-3.3-Nemotron-Super-49B-Healthcare-Text2sql-v1.0	`nvidia/llama-3.3-nemotron-super-49b-healthcare-text2sql-v1.0`
MiniMax-M2.5	`minimax-ai/minimax-m25`
NVIDIA-Nemotron-Nano-9B-v2-DGX-Spark	`nvidia/nvidia-nemotron-nano-9b-v2-dgx-spark`
Nemotron-3-Super-120B-A12B	`nvidia/nemotron-3-super-120b-a12b`
Qwen3-Coder-Next	`qwen/qwen3-coder-next`
Qwen3-Next-80B-A3B-Instruct	`qwen/qwen3-next-80b-a3b-instruct`
Qwen3 Next 80B A3B Thinking	`qwen/qwen3-next-80b-a3b-thinking`
Qwen3-32B	`qwen/qwen3-32b`
Qwen3-32B NIM for DGX Spark	`qwen/qwen3-32b-dgx-spark`
Riva-Translate-4b-Instruct-v1.1	`nvidia/riva-translate-4b-instruct-v1.1`