Platform Support
NVIDIA H100
NVIDIA A100 80 GB
NVIDIA L40S
The GPUs must be installed in an NVIDIA AI Enterprise qualified system. Refer to the NVIDIA Qualified System Catalog more information.
The GPU memory requirements vary according to the size of the model and the number of GPUs in a system. Refer to the NVIDIA MeMo Inference Microservice Support Matrix and the NVIDIA NeMo Retriever Embedding Microservice Support Matrix for more information.
NVIDIA vGPU 17
Operating System |
Kubernetes |
VMware vSphere with Tanzu |
|---|---|---|
| Ubuntu 22.04 | 1.26—1.28 | 8.0 Update 2 |
Operating System |
containerd |
|---|---|
| Ubuntu 22.04 | 1.6, 1.7 |