Is this page helpful?

Support Matrix#

Hardware#

Unless specified otherwise, NVIDIA NIM for vision language models (VLMs) should, but are not guaranteed to, run on any NVIDIA GPU, provided the GPU has sufficient memory. They can also run on multiple homogeneous NVIDIA GPUs with sufficient aggregate memory and a CUDA compute capability of >= 7.0 (8.0 for bfloat16) unless otherwise specified. For more information, refer to Supported Models.

NVIDIA NIM for VLMs does not support NVIDIA Virtual GPU (vGPU) environments.

For information on the supported operating systems, drivers, and software, refer to the About Get Started page.

Supported Models#

Supported Model-Free NIM Images#

The following model-free NIM images are available:

  • nvcr.io/nim/nvidia/vllm-model-free-nim:2.1.0 is the vLLM Model-Free NIM for VLMs using the vLLM backend.

  • nvcr.io/nim/nvidia/sglang-model-free-nim:2.1.0 is the SGLang Model-Free NIM for VLMs using the SGLang backend.

The vLLM model-free NIM is supported on GPUs with a compute capability of 7.5 or higher.

The SGLang model-free NIM is supported on the following GPUs:

  • NVIDIA-H200-SXM-141GB

  • NVIDIA-B200-180GB

  • NVIDIA-H20-3e-141GB

For more information, refer to Model-Free NIM.