Platform Support#
This page outlines the supported hardware and storage requirements for the NVIDIA NIM Operator and its supported microservices.
NVIDIA NIM Microservices#
The NIM Operator supports deployment of NVIDIA NIM across various domains such as reasoning, retrieval, speech, and biology. For a full list of supported NIMs, refer to the build.nvidia.com. Refer to documentation for a specific NIM to learn more about its features and capabilities.
NIM Microservice System Requirements#
The NVIDIA NIM Operator recommends that you have access to NVIDIA GPUs such as NVIDIA A100 80 GB, H100, or L40S GPUs on one or more nodes to be able to run models. Your specific platform requirements depend on the models you are using. For example, large models may exceed the memory capacity of one GPU, meaning you will need to add more GPUs to your cluster.
When you deploy a NIM Service or NIM pipeline, you can specify more than one GPU for a workload. You can download your models using NIM Cache.
You can view supported models and their resource requirements on build.nvidia.com. Refer to the model details on build.nvidia.com for specific details on storage, GPU model, and GPU count for the NIVIDA NIM you want to use.
To find details on supported models on build.nvidia.com:
Use the Filter by text to search for
NVIDIA NIM
models and selectRun Anywhere
from the NIM Type drop down menu. Use the additional filter options to find the exact model you want.Click on the model you want to use.
Click Model Card to view details about the NIM deployment requirements.
NVIDIA NeMo Microservices#
NVIDIA NIM Operator supports the following NeMo microservices.
Microservice |
Version |
---|---|
NVIDIA NeMo Data Store |
v25.4.0 |
NVIDIA NeMo Entity Store |
v25.4.0 |
NVIDIA NeMo Guardrails |
v25.4.0 |
NVIDIA NeMo Evaluator |
v25.4.0 |
NVIDIA NeMo Customizer |
v25.4.0 |
NeMo Microservices System Requirements#
Refer to the NeMo Requirements documentation for full details on system requirements.
Operating Systems and Kubernetes Platforms#
NVIDIA NIM Operator requires NVIDIA GPU Operator version 24.3.0 or higher to be installed on your cluster. Refer to Supported Operating Systems and Kubernetes Platforms in the GPU Operator documentation for supported operating systems and Kubernetes platforms.
Command-Line Tools#
Tool |
Installation Documentation |
---|---|
kubectl (version matching your Kubernetes cluster) |
Refer to Install Tools in the Kubernetes documentation for more information. |
Helm v3 and higher |
Refer to Install Helm in the Helm documentation for more information. |