Platform Support#

This page outlines the supported hardware and storage requirements for the NVIDIA NIM Operator and its supported microservices.

NVIDIA NIM Microservices#

The NIM Operator supports deployment of NVIDIA NIM across various domains such as reasoning, retrieval, speech, and biology. For a full list of supported NIMs, refer to the build.nvidia.com. Refer to documentation for a specific NIM to learn more about its features and capabilities.

NIM Microservice System Requirements#

The NVIDIA NIM Operator recommends that you have access to NVIDIA GPUs such as NVIDIA A100 80 GB, H100, or L40S GPUs on one or more nodes to be able to run models. Your specific platform requirements depend on the models you are using. For example, large models may exceed the memory capacity of one GPU, meaning you will need to add more GPUs to your cluster.

When you deploy a NIM Service or NIM pipeline, you can specify more than one GPU for a workload. You can download your models using NIM Cache.

You can view supported models and their resource requirements on build.nvidia.com. Refer to the model details on build.nvidia.com for specific details on storage, GPU model, and GPU count for the NIVIDA NIM you want to use.

To find details on supported models on build.nvidia.com:

  1. Use the Filter by text to search for NVIDIA NIM models and select Run Anywhere from the NIM Type drop down menu. Use the additional filter options to find the exact model you want.

  2. Click on the model you want to use.

  3. Click Model Card to view details about the NIM deployment requirements.

NVIDIA NeMo Microservices#

NVIDIA NIM Operator supports the following NeMo microservices.

Microservice

Version

NVIDIA NeMo Data Store

v25.4.0

NVIDIA NeMo Entity Store

v25.4.0

NVIDIA NeMo Guardrails

v25.4.0

NVIDIA NeMo Evaluator

v25.4.0

NVIDIA NeMo Customizer

v25.4.0

NeMo Microservices System Requirements#

Refer to the NeMo Requirements documentation for full details on system requirements.

Operating Systems and Kubernetes Platforms#

NVIDIA NIM Operator requires NVIDIA GPU Operator version 24.3.0 or higher to be installed on your cluster. Refer to Supported Operating Systems and Kubernetes Platforms in the GPU Operator documentation for supported operating systems and Kubernetes platforms.

Command-Line Tools#

Tool

Installation Documentation

kubectl (version matching your Kubernetes cluster)

Refer to Install Tools in the Kubernetes documentation for more information.

Helm v3 and higher

Refer to Install Helm in the Helm documentation for more information.