Support Matrix#
Supported Hardware#
Minimum System Hardware Requirements#
CPU Cores |
RAM (GB) |
NVMe SSD Storage (GB) |
GPU Count |
---|---|---|---|
12 |
64 |
48 |
One or More NVIDIA GPU from the list of Supported NVIDIA GPUs |
Supported NVIDIA GPUs#
The Boltz-2 NIM requires NVIDIA GPUs with at least 48 GB of GPU Memory. While the system can run on any NVIDIA GPU meeting this minimum memory requirement, the following configurations have been specifically tested and optimized using NVIDIA TensorRT (TRT):
GPU |
GPU Memory (GB) |
Precision |
GPU Count |
---|---|---|---|
A100 |
80 |
BF16 |
1-2-4-8 |
H100 |
80 |
BF16 |
1-2-4-8 |
L40s |
48 |
BF16 |
1-2-4-8 |
For optimal performance and stability, we recommend using one of the above tested configurations. While other NVIDIA GPUs with 48GB or more memory may work, they have not been officially tested and may exhibit degraded performance or stability issues.
Testing Locally Available Hardware#
Begin with Docker-supported operating system
Install Docker - minimum version: 23.0.1
Install NVIDIA Drivers - minimum version: 535
Install the NVIDIA Container Toolkit - minimum version: 1.13.5
Verify your container runtime supports NVIDIA GPUs by running
docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi
Example output:
+-----------------------------------------------------------------------------+ | NVIDIA-SMI 535.230.02 Driver Version: 535.230.02 CUDA Version: 12.4 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 NVIDIA GeForce ... Off | 00000000:01:00.0 Off | N/A | | 41% 30C P8 1W / 260W | 2244MiB / 11264MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| +-----------------------------------------------------------------------------+
Note
For more information on enumerating multi-GPU systems, refer to the NVIDIA Container Toolkit’s GPU Enumeration Docs