Support Matrix#

Supported Hardware#

Minimum System Hardware Requirements#

CPU Cores	RAM (GB)	NVMe SSD Storage (GB)	GPU Count
12	64	48	One or More NVIDIA GPU from the list of Supported NVIDIA GPUs

Supported NVIDIA GPUs#

The Boltz-2 NIM supports NVIDIA GPUs with at least 48 GB of GPU Memory. The following configurations are optimized using NVIDIA TensorRT (TRT) for specific GPU models:

GPU	GPU Memory (GB)	Precision	GPU Count
A100	80	BF16	1-2-4-8
H100	80	BF16	1-2-4-8
L40s	48	BF16	1-2-4-8

While the NIM will run on any NVIDIA GPU with at least 48 GB of memory, performance and stability may be degraded on system configurations other than the above.

Testing Locally Available Hardware#

Begin with Docker-supported operating system
Install Docker - minimum version: 23.0.1
Install NVIDIA Drivers - minimum version: 535
Install the NVIDIA Container Toolkit - minimum version: 1.13.5
Verify your container runtime supports NVIDIA GPUs by running

docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

Example output:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 535.230.02   Driver Version: 535.230.02    CUDA Version: 12.4    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| 41%   30C    P8     1W / 260W |   2244MiB / 11264MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

Note

For more information on enumerating multi-GPU systems, refer to the NVIDIA Container Toolkit’s GPU Enumeration Docs