Support Matrix for Certified NIMs

This page lists the supported models, their deployment profiles, and the verified hardware SKUs for NIM LLM Certified NIMs. For NIM Day 0, refer to Support Matrix for NIM Day 0. For NIM Turbo, refer to Support Matrix for NIM Turbo.

Supported Models and Profiles

Use the following sections to identify the supported deployment profiles for each model. Profile strings follow a naming convention described in Model Profiles and Selection.

Use the table below to filter certified NIM profiles by GPU, tensor parallelism (TP), precision, and model name. Each row is one supported profile; details also appear in the per-model sections that follow.

GPU All GPUs NVIDIA-A100-80GB-PCIe NVIDIA-A100-PCIE-40GB NVIDIA-A100-SXM4-40GB NVIDIA-A100-SXM4-80GB NVIDIA-A10G NVIDIA-B200 NVIDIA-B300-SXM6-AC NVIDIA-GB10 NVIDIA-GB200 NVIDIA-GB300 NVIDIA-GH200-144G-HBM3e NVIDIA-GH200-480GB NVIDIA-H100-80GB-HBM3 NVIDIA-H100-NVL NVIDIA-H100-PCIe NVIDIA-H200 NVIDIA-H200-NVL NVIDIA-L40S NVIDIA-RTX-PRO-4500-Blackwell-Server-Edition NVIDIA-RTX-PRO-6000-Blackwell-Server-Edition TP All TP 1 2 4 8 Precision All Precisions BF16 FP8 MXFP4 NVFP4 LoRA only Search model

Model	TP	Precision	LoRA
gpt-oss-120b	1	MXFP4	No
gpt-oss-120b	2	MXFP4	No
gpt-oss-120b	4	MXFP4	No
gpt-oss-120b	8	MXFP4	No
gpt-oss-120b	1	MXFP4	Yes
gpt-oss-120b	2	MXFP4	Yes
gpt-oss-120b	4	MXFP4	Yes
gpt-oss-120b	8	MXFP4	Yes
gpt-oss-20b	1	MXFP4	No
gpt-oss-20b	2	MXFP4	No
gpt-oss-20b	4	MXFP4	No
gpt-oss-20b	8	MXFP4	No
gpt-oss-20b	1	MXFP4	Yes
gpt-oss-20b	2	MXFP4	Yes
gpt-oss-20b	4	MXFP4	Yes
gpt-oss-20b	8	MXFP4	Yes
llama-3.1-70b-instruct	1	BF16	No
llama-3.1-70b-instruct	2	BF16	No
llama-3.1-70b-instruct	4	BF16	No
llama-3.1-70b-instruct	8	BF16	No
llama-3.1-70b-instruct	1	BF16	Yes
llama-3.1-70b-instruct	2	BF16	Yes
llama-3.1-70b-instruct	4	BF16	Yes
llama-3.1-70b-instruct	8	BF16	Yes
llama-3.1-70b-instruct	1	FP8	No
llama-3.1-70b-instruct	2	FP8	No
llama-3.1-70b-instruct	4	FP8	No
llama-3.1-70b-instruct	8	FP8	No
llama-3.1-70b-instruct	1	FP8	Yes
llama-3.1-70b-instruct	2	FP8	Yes
llama-3.1-70b-instruct	4	FP8	Yes
llama-3.1-70b-instruct	8	FP8	Yes
llama-3.1-70b-instruct	1	NVFP4	No
llama-3.1-70b-instruct	2	NVFP4	No
llama-3.1-70b-instruct	4	NVFP4	No
llama-3.1-70b-instruct	8	NVFP4	No
llama-3.1-70b-instruct	1	NVFP4	Yes
llama-3.1-70b-instruct	2	NVFP4	Yes
llama-3.1-70b-instruct	4	NVFP4	Yes
llama-3.1-70b-instruct	8	NVFP4	Yes
llama-3.1-8b-instruct	1	BF16	No
llama-3.1-8b-instruct	1	BF16	Yes
llama-3.1-8b-instruct	1	FP8	No
llama-3.1-8b-instruct	1	FP8	Yes
llama-3.1-8b-instruct	1	NVFP4	No
llama-3.1-8b-instruct	1	NVFP4	Yes
llama-3.3-70b-instruct	1	BF16	No
llama-3.3-70b-instruct	2	BF16	No
llama-3.3-70b-instruct	4	BF16	No
llama-3.3-70b-instruct	8	BF16	No
llama-3.3-70b-instruct	1	BF16	Yes
llama-3.3-70b-instruct	2	BF16	Yes
llama-3.3-70b-instruct	4	BF16	Yes
llama-3.3-70b-instruct	8	BF16	Yes
llama-3.3-70b-instruct	1	FP8	No
llama-3.3-70b-instruct	2	FP8	No
llama-3.3-70b-instruct	4	FP8	No
llama-3.3-70b-instruct	8	FP8	No
llama-3.3-70b-instruct	1	FP8	Yes
llama-3.3-70b-instruct	2	FP8	Yes
llama-3.3-70b-instruct	4	FP8	Yes
llama-3.3-70b-instruct	8	FP8	Yes
llama-3.3-70b-instruct	1	NVFP4	No
llama-3.3-70b-instruct	2	NVFP4	No
llama-3.3-70b-instruct	4	NVFP4	No
llama-3.3-70b-instruct	8	NVFP4	No
llama-3.3-70b-instruct	1	NVFP4	Yes
llama-3.3-70b-instruct	2	NVFP4	Yes
llama-3.3-70b-instruct	4	NVFP4	Yes
llama-3.3-70b-instruct	8	NVFP4	Yes
llama-3.3-nemotron-super-49b-v1.5	1	BF16	No
llama-3.3-nemotron-super-49b-v1.5	2	BF16	No
llama-3.3-nemotron-super-49b-v1.5	4	BF16	No
llama-3.3-nemotron-super-49b-v1.5	8	BF16	No
llama-3.3-nemotron-super-49b-v1.5	1	BF16	Yes
llama-3.3-nemotron-super-49b-v1.5	2	BF16	Yes
llama-3.3-nemotron-super-49b-v1.5	4	BF16	Yes
llama-3.3-nemotron-super-49b-v1.5	8	BF16	Yes
llama-3.3-nemotron-super-49b-v1.5	1	FP8	No
llama-3.3-nemotron-super-49b-v1.5	2	FP8	No
llama-3.3-nemotron-super-49b-v1.5	4	FP8	No
llama-3.3-nemotron-super-49b-v1.5	8	FP8	No
llama-3.3-nemotron-super-49b-v1.5	1	FP8	Yes
llama-3.3-nemotron-super-49b-v1.5	2	FP8	Yes
llama-3.3-nemotron-super-49b-v1.5	4	FP8	Yes
llama-3.3-nemotron-super-49b-v1.5	8	FP8	Yes
llama-3.3-nemotron-super-49b-v1.5	1	NVFP4	No
llama-3.3-nemotron-super-49b-v1.5	2	NVFP4	No
llama-3.3-nemotron-super-49b-v1.5	4	NVFP4	No
llama-3.3-nemotron-super-49b-v1.5	8	NVFP4	No
llama-3.3-nemotron-super-49b-v1.5	1	NVFP4	Yes
llama-3.3-nemotron-super-49b-v1.5	2	NVFP4	Yes
llama-3.3-nemotron-super-49b-v1.5	4	NVFP4	Yes
llama-3.3-nemotron-super-49b-v1.5	8	NVFP4	Yes
nemotron-3-nano	1	BF16	No
nemotron-3-nano	2	BF16	No
nemotron-3-nano	4	BF16	No
nemotron-3-nano	8	BF16	No
nemotron-3-nano	1	BF16	Yes
nemotron-3-nano	2	BF16	Yes
nemotron-3-nano	4	BF16	Yes
nemotron-3-nano	8	BF16	Yes
nemotron-3-nano	1	FP8	No
nemotron-3-nano	2	FP8	No
nemotron-3-nano	4	FP8	No
nemotron-3-nano	8	FP8	No
nemotron-3-nano	1	FP8	Yes
nemotron-3-nano	2	FP8	Yes
nemotron-3-nano	4	FP8	Yes
nemotron-3-nano	8	FP8	Yes
nemotron-3-nano	1	NVFP4	No
nemotron-3-nano	2	NVFP4	No
nemotron-3-nano	4	NVFP4	No
nemotron-3-nano	8	NVFP4	No
nemotron-3-super-120b-a12b	1	BF16	No
nemotron-3-super-120b-a12b	2	BF16	No
nemotron-3-super-120b-a12b	4	BF16	No
nemotron-3-super-120b-a12b	8	BF16	No
nemotron-3-super-120b-a12b	1	BF16	Yes
nemotron-3-super-120b-a12b	2	BF16	Yes
nemotron-3-super-120b-a12b	4	BF16	Yes
nemotron-3-super-120b-a12b	8	BF16	Yes
nemotron-3-super-120b-a12b	1	FP8	No
nemotron-3-super-120b-a12b	2	FP8	No
nemotron-3-super-120b-a12b	4	FP8	No
nemotron-3-super-120b-a12b	8	FP8	No
nemotron-3-super-120b-a12b	1	FP8	Yes
nemotron-3-super-120b-a12b	2	FP8	Yes
nemotron-3-super-120b-a12b	4	FP8	Yes
nemotron-3-super-120b-a12b	8	FP8	Yes
nemotron-3-super-120b-a12b	1	NVFP4	No
nemotron-3-super-120b-a12b	2	NVFP4	No
nemotron-3-super-120b-a12b	4	NVFP4	No
nemotron-3-super-120b-a12b	8	NVFP4	No
nemotron-3-super-120b-a12b	1	NVFP4	Yes
nemotron-3-super-120b-a12b	2	NVFP4	Yes
nemotron-3-super-120b-a12b	4	NVFP4	Yes
nemotron-3-super-120b-a12b	8	NVFP4	Yes
starcoder2-7b	1	BF16	No
starcoder2-7b	2	BF16	No
No matching certified profiles. This configuration may still be deployable with memory tuning; refer to the memory troubleshooting guide for details.
No matching certified profiles. This GPU is verified for Model-Free NIM deployment, and this configuration may also be deployable with memory tuning; refer to the memory troubleshooting guide for details.

gpt-oss-120b

Latest supported NIM LLM version: 2.0.4

The following table lists the supported profile configurations for openai/gpt-oss-120b:

Precision	TP1	TP2	TP4	TP8
MXFP4	vllm-mxfp4-tp1-pp1	vllm-mxfp4-tp2-pp1	vllm-mxfp4-tp4-pp1	vllm-mxfp4-tp8-pp1
MXFP4 + LoRA	vllm-mxfp4-tp1-pp1-lora	vllm-mxfp4-tp2-pp1-lora	vllm-mxfp4-tp4-pp1-lora	vllm-mxfp4-tp8-pp1-lora

Verified GPUs

This model has been verified on the following GPUs:

• NVIDIA-A100-SXM4-40GB

• NVIDIA-A100-SXM4-80GB

• NVIDIA-A10G

• NVIDIA-B200

• NVIDIA-B300-SXM6-AC

• NVIDIA-GB200

• NVIDIA-GB300

• NVIDIA-GH200-144G-HBM3e

• NVIDIA-GH200-480GB

• NVIDIA-H100-80GB-HBM3

• NVIDIA-H100-NVL

• NVIDIA-H200

• NVIDIA-H200-NVL

• NVIDIA-L40S

• NVIDIA-RTX-PRO-4500-Blackwell-Server-Edition

• NVIDIA-RTX-PRO-6000-Blackwell-Server-Edition

gpt-oss-20b

Latest supported NIM LLM version: 2.0.4

The following table lists the supported profile configurations for openai/gpt-oss-20b:

Precision	TP1	TP2	TP4	TP8
MXFP4	vllm-mxfp4-tp1-pp1	vllm-mxfp4-tp2-pp1	vllm-mxfp4-tp4-pp1	vllm-mxfp4-tp8-pp1
MXFP4 + LoRA	vllm-mxfp4-tp1-pp1-lora	vllm-mxfp4-tp2-pp1-lora	vllm-mxfp4-tp4-pp1-lora	vllm-mxfp4-tp8-pp1-lora

Verified GPUs

This model has been verified on the following GPUs:

• NVIDIA-A100-SXM4-40GB

• NVIDIA-A100-SXM4-80GB

• NVIDIA-A10G

• NVIDIA-B200

• NVIDIA-B300-SXM6-AC

• NVIDIA-GB10

• NVIDIA-GB200

• NVIDIA-GB300

• NVIDIA-GH200-144G-HBM3e

• NVIDIA-GH200-480GB

• NVIDIA-H100-80GB-HBM3

• NVIDIA-H100-NVL

• NVIDIA-H200

• NVIDIA-H200-NVL

• NVIDIA-L40S

• NVIDIA-RTX-PRO-4500-Blackwell-Server-Edition

• NVIDIA-RTX-PRO-6000-Blackwell-Server-Edition

llama-3.1-70b-instruct

Latest supported NIM LLM version: 2.0.4

The following table lists the supported profile configurations for meta/llama-3.1-70b-instruct:

Precision	TP1	TP2	TP4	TP8
BF16	vllm-bf16-tp1-pp1	vllm-bf16-tp2-pp1	vllm-bf16-tp4-pp1	vllm-bf16-tp8-pp1
BF16 + LoRA	vllm-bf16-tp1-pp1-lora	vllm-bf16-tp2-pp1-lora	vllm-bf16-tp4-pp1-lora	vllm-bf16-tp8-pp1-lora
FP8	vllm-fp8-tp1-pp1	vllm-fp8-tp2-pp1	vllm-fp8-tp4-pp1	vllm-fp8-tp8-pp1
FP8 + LoRA	vllm-fp8-tp1-pp1-lora	vllm-fp8-tp2-pp1-lora	vllm-fp8-tp4-pp1-lora	vllm-fp8-tp8-pp1-lora
NVFP4	vllm-nvfp4-tp1-pp1	vllm-nvfp4-tp2-pp1	vllm-nvfp4-tp4-pp1	vllm-nvfp4-tp8-pp1
NVFP4 + LoRA	vllm-nvfp4-tp1-pp1-lora	vllm-nvfp4-tp2-pp1-lora	vllm-nvfp4-tp4-pp1-lora	vllm-nvfp4-tp8-pp1-lora

Verified GPUs

This model has been verified on the following GPUs:

• NVIDIA-A10G

• NVIDIA-A100-SXM4-40GB

• NVIDIA-A100-SXM4-80GB

• NVIDIA-B200

• NVIDIA-B300-SXM6-AC

• NVIDIA-GB200

• NVIDIA-GB300

• NVIDIA-GH200-144G-HBM3e

• NVIDIA-GH200-480GB

• NVIDIA-H100-80GB-HBM3

• NVIDIA-H100-NVL

• NVIDIA-H200

• NVIDIA-H200-NVL

• NVIDIA-L40S

• NVIDIA-RTX-PRO-4500-Blackwell-Server-Edition

• NVIDIA-RTX-PRO-6000-Blackwell-Server-Edition

llama-3.1-8b-instruct

Latest supported NIM LLM version: 2.0.4

The following table lists the supported profile configurations for meta/llama-3.1-8b-instruct:

Precision	TP1
BF16	vllm-bf16-tp1-pp1
BF16 + LoRA	vllm-bf16-tp1-pp1-lora
FP8	vllm-fp8-tp1-pp1
FP8 + LoRA	vllm-fp8-tp1-pp1-lora
NVFP4	vllm-nvfp4-tp1-pp1
NVFP4 + LoRA	vllm-nvfp4-tp1-pp1-lora

Verified GPUs

This model has been verified on the following GPUs:

• NVIDIA-A100-SXM4-40GB

• NVIDIA-A100-SXM4-80GB

• NVIDIA-A10G

• NVIDIA-B200

• NVIDIA-B300-SXM6-AC

• NVIDIA-GB10

• NVIDIA-GB200

• NVIDIA-GB300

• NVIDIA-GH200-144G-HBM3e

• NVIDIA-GH200-480GB

• NVIDIA-H100-80GB-HBM3

• NVIDIA-H100-NVL

• NVIDIA-H200

• NVIDIA-H200-NVL

• NVIDIA-L40S

• NVIDIA-RTX-PRO-4500-Blackwell-Server-Edition

• NVIDIA-RTX-PRO-6000-Blackwell-Server-Edition

llama-3.3-70b-instruct

Latest supported NIM LLM version: 2.0.4

The following table lists the supported profile configurations for meta/llama-3.3-70b-instruct:

Precision	TP1	TP2	TP4	TP8
BF16	vllm-bf16-tp1-pp1	vllm-bf16-tp2-pp1	vllm-bf16-tp4-pp1	vllm-bf16-tp8-pp1
BF16 + LoRA	vllm-bf16-tp1-pp1-lora	vllm-bf16-tp2-pp1-lora	vllm-bf16-tp4-pp1-lora	vllm-bf16-tp8-pp1-lora
FP8	vllm-fp8-tp1-pp1	vllm-fp8-tp2-pp1	vllm-fp8-tp4-pp1	vllm-fp8-tp8-pp1
FP8 + LoRA	vllm-fp8-tp1-pp1-lora	vllm-fp8-tp2-pp1-lora	vllm-fp8-tp4-pp1-lora	vllm-fp8-tp8-pp1-lora
NVFP4	vllm-nvfp4-tp1-pp1	vllm-nvfp4-tp2-pp1	vllm-nvfp4-tp4-pp1	vllm-nvfp4-tp8-pp1
NVFP4 + LoRA	vllm-nvfp4-tp1-pp1-lora	vllm-nvfp4-tp2-pp1-lora	vllm-nvfp4-tp4-pp1-lora	vllm-nvfp4-tp8-pp1-lora

Verified GPUs

This model has been verified on the following GPUs:

• NVIDIA-A10G

• NVIDIA-A100-SXM4-40GB

• NVIDIA-A100-SXM4-80GB

• NVIDIA-B200

• NVIDIA-B300-SXM6-AC

• NVIDIA-GB200

• NVIDIA-GB300

• NVIDIA-GH200-144G-HBM3e

• NVIDIA-GH200-480GB

• NVIDIA-H100-80GB-HBM3

• NVIDIA-H100-NVL

• NVIDIA-H200

• NVIDIA-H200-NVL

• NVIDIA-L40S

• NVIDIA-RTX-PRO-4500-Blackwell-Server-Edition

• NVIDIA-RTX-PRO-6000-Blackwell-Server-Edition

llama-3.3-nemotron-super-49b-v1.5

Latest supported NIM LLM version: 2.0.4

The following table lists the supported profile configurations for nvidia/llama-3.3-nemotron-super-49b-v1.5:

Precision	TP1	TP2	TP4	TP8
BF16	vllm-bf16-tp1-pp1	vllm-bf16-tp2-pp1	vllm-bf16-tp4-pp1	vllm-bf16-tp8-pp1
BF16 + LoRA	vllm-bf16-tp1-pp1-lora	vllm-bf16-tp2-pp1-lora	vllm-bf16-tp4-pp1-lora	vllm-bf16-tp8-pp1-lora
FP8	vllm-fp8-tp1-pp1	vllm-fp8-tp2-pp1	vllm-fp8-tp4-pp1	vllm-fp8-tp8-pp1
FP8 + LoRA	vllm-fp8-tp1-pp1-lora	vllm-fp8-tp2-pp1-lora	vllm-fp8-tp4-pp1-lora	vllm-fp8-tp8-pp1-lora
NVFP4	vllm-nvfp4-tp1-pp1	vllm-nvfp4-tp2-pp1	vllm-nvfp4-tp4-pp1	vllm-nvfp4-tp8-pp1
NVFP4 + LoRA	vllm-nvfp4-tp1-pp1-lora	vllm-nvfp4-tp2-pp1-lora	vllm-nvfp4-tp4-pp1-lora	vllm-nvfp4-tp8-pp1-lora

Verified GPUs

This model has been verified on the following GPUs:

• NVIDIA-A10G

• NVIDIA-A100-SXM4-40GB

• NVIDIA-A100-SXM4-80GB

• NVIDIA-B200

• NVIDIA-B300-SXM6-AC

• NVIDIA-GB10

• NVIDIA-GB200

• NVIDIA-GB300

• NVIDIA-GH200-144G-HBM3e

• NVIDIA-GH200-480GB

• NVIDIA-H100-80GB-HBM3

• NVIDIA-H100-NVL

• NVIDIA-H200

• NVIDIA-H200-NVL

• NVIDIA-L40S

• NVIDIA-RTX-PRO-4500-Blackwell-Server-Edition

• NVIDIA-RTX-PRO-6000-Blackwell-Server-Edition

nemotron-3-nano

Latest supported NIM LLM version: 2.0.4

The following table lists the supported profile configurations for nvidia/nemotron-3-nano:

Precision	TP1	TP2	TP4	TP8
BF16	vllm-bf16-tp1-pp1	vllm-bf16-tp2-pp1	vllm-bf16-tp4-pp1	vllm-bf16-tp8-pp1
BF16 + LoRA	vllm-bf16-tp1-pp1-lora	vllm-bf16-tp2-pp1-lora	vllm-bf16-tp4-pp1-lora	vllm-bf16-tp8-pp1-lora
FP8	vllm-fp8-tp1-pp1	vllm-fp8-tp2-pp1	vllm-fp8-tp4-pp1	vllm-fp8-tp8-pp1
FP8 + LoRA	vllm-fp8-tp1-pp1-lora	vllm-fp8-tp2-pp1-lora	vllm-fp8-tp4-pp1-lora	vllm-fp8-tp8-pp1-lora
NVFP4	vllm-nvfp4-tp1-pp1	vllm-nvfp4-tp2-pp1	vllm-nvfp4-tp4-pp1	vllm-nvfp4-tp8-pp1

Verified GPUs

This model has been verified on the following GPUs:

• NVIDIA-A100-SXM4-40GB

• NVIDIA-A100-SXM4-80GB

• NVIDIA-A10G

• NVIDIA-B200

• NVIDIA-B300-SXM6-AC

• NVIDIA-GB10

• NVIDIA-GB200

• NVIDIA-GB300

• NVIDIA-GH200-144G-HBM3e

• NVIDIA-GH200-480GB

• NVIDIA-H100-80GB-HBM3

• NVIDIA-H100-NVL

• NVIDIA-H200

• NVIDIA-H200-NVL

• NVIDIA-L40S

• NVIDIA-RTX-PRO-4500-Blackwell-Server-Edition

• NVIDIA-RTX-PRO-6000-Blackwell-Server-Edition

nemotron-3-super-120b-a12b

Latest supported NIM LLM version: 2.0.4

The following table lists the supported profile configurations for nvidia/nemotron-3-super-120b-a12b:

Precision	TP1	TP2	TP4	TP8
BF16	vllm-bf16-tp1-pp1	vllm-bf16-tp2-pp1	vllm-bf16-tp4-pp1	vllm-bf16-tp8-pp1
BF16 + LoRA	vllm-bf16-tp1-pp1-lora	vllm-bf16-tp2-pp1-lora	vllm-bf16-tp4-pp1-lora	vllm-bf16-tp8-pp1-lora
FP8	vllm-fp8-tp1-pp1	vllm-fp8-tp2-pp1	vllm-fp8-tp4-pp1	vllm-fp8-tp8-pp1
FP8 + LoRA	vllm-fp8-tp1-pp1-lora	vllm-fp8-tp2-pp1-lora	vllm-fp8-tp4-pp1-lora	vllm-fp8-tp8-pp1-lora
NVFP4	vllm-nvfp4-tp1-pp1	vllm-nvfp4-tp2-pp1	vllm-nvfp4-tp4-pp1	vllm-nvfp4-tp8-pp1
NVFP4 + LoRA	vllm-nvfp4-tp1-pp1-lora	vllm-nvfp4-tp2-pp1-lora	vllm-nvfp4-tp4-pp1-lora	vllm-nvfp4-tp8-pp1-lora

Note

This is a large model. Lower-TP profiles require substantially more GPU memory per device, so some verified GPUs support only TP4 or TP8 profiles.

Verified GPUs

The following GPUs have been verified with one or more supported profiles for this model:

• NVIDIA-A100-SXM4-40GB

• NVIDIA-A100-SXM4-80GB

• NVIDIA-B200

• NVIDIA-B300-SXM6-AC

• NVIDIA-GB200

• NVIDIA-GB300

• NVIDIA-GH200-144G-HBM3e

• NVIDIA-H100-80GB-HBM3

• NVIDIA-H100-NVL

• NVIDIA-H200

• NVIDIA-H200-NVL

• NVIDIA-L40S

• NVIDIA-RTX-PRO-4500-Blackwell-Server-Edition

• NVIDIA-RTX-PRO-6000-Blackwell-Server-Edition

starcoder2-7b

Latest supported NIM LLM version: 2.0.4

The following table lists the supported profile configurations for bigcode/starcoder2-7b:

Precision	TP1	TP2
BF16	vllm-bf16-tp1-pp1	vllm-bf16-tp2-pp1

Verified GPUs

This model has been verified on the following GPUs:

• NVIDIA-H100-80GB-HBM3

• NVIDIA-H200

Model-Free NIM

Latest supported NIM LLM version: 2.0.4

The following models are tested and validated for nvidia/model-free-nim:

• gpt-oss-20b

• apriel-nemotron

• codestral

While not explicitly validated, the model-free NIM can be used with any model supported by the underlying backend (vLLM) version. Refer to Model-Free NIM for deployment details.

Verified GPUs

The model-free NIM has been verified on the following GPUs:

• NVIDIA-A100-80GB-PCIe

• NVIDIA-A100-PCIE-40GB

• NVIDIA-A100-SXM4-40GB

• NVIDIA-A100-SXM4-80GB

• NVIDIA-A10G

• NVIDIA-B200

• NVIDIA-B300-SXM6-AC

• NVIDIA-GB10

• NVIDIA-GB200

• NVIDIA-GB300

• NVIDIA-GH200-144G-HBM3e

• NVIDIA-GH200-480GB

• NVIDIA-H100-80GB-HBM3

• NVIDIA-H100-NVL

• NVIDIA-H100-PCIe

• NVIDIA-H200

• NVIDIA-H200-NVL

• NVIDIA-L40S

• NVIDIA-RTX-PRO-4500-Blackwell-Server-Edition

• NVIDIA-RTX-PRO-6000-Blackwell-Server-Edition

1.x NIM LLM Models

For more information on version 1.x NIMs, refer to the 1.15 version of the NIM LLM Supported Models page.

Show 1.x models

Model (Hardware Requirements)	Organization/Model ID (Catalog Page)
DeepSeek-V3.1-Terminus	deepseek-ai/deepseek-v3.1-terminus
DeepSeek-V3.2-Exp	deepseek-ai/deepseek-v32-exp-nim
GLM-5	zai-org/glm-5
GLM-5.1	zai-org/glm-51
Llama-3.1-Nemotron-Nano-8B-Healthcare-Text2sql-v1.0	nvidia/llama-3.1-nemotron-nano-8b-healthcare-text2sql-v1.0
Llama-3.3-Nemotron-Super-49B-Healthcare-Text2sql-v1.0	nvidia/llama-3.3-nemotron-super-49b-healthcare-text2sql-v1.0
MiniMax-M2.5	minimax-ai/minimax-m25
NVIDIA-Nemotron-Nano-9B-v2-DGX-Spark	nvidia/nvidia-nemotron-nano-9b-v2-dgx-spark
Nemotron-3-Super-120B-A12B	nvidia/nemotron-3-super-120b-a12b
Qwen3-Coder-Next	qwen/qwen3-coder-next
Qwen3-Next-80B-A3B-Instruct	qwen/qwen3-next-80b-a3b-instruct
Qwen3 Next 80B A3B Thinking	qwen/qwen3-next-80b-a3b-thinking
Qwen3-32B	qwen/qwen3-32b
Qwen3-32B NIM for DGX Spark	qwen/qwen3-32b-dgx-spark
Riva-Translate-4b-Instruct-v1.1	nvidia/riva-translate-4b-instruct-v1.1