Support Matrix#
Supported Hardware#
Minimum System Hardware Requirements#
CPU Cores |
RAM (GB) |
NVMe SSD Storage (GB) |
GPU Count |
|---|---|---|---|
12 |
64 |
80 |
One or More NVIDIA GPU from the list of Supported NVIDIA GPUs |
Note
The NVMe SSD Storage requirement of 80GB accounts for the Boltz-2 NIM image size (~73GB disk usage) plus additional space for Docker layer caching and temporary files during runtime.
Supported NVIDIA GPUs#
The Boltz-2 NIM requires NVIDIA GPUs with at least 48 GB of GPU Memory. While the system can run on any NVIDIA GPU meeting this minimum memory requirement, the following configurations have been specifically tested and optimized using NVIDIA TensorRT (TRT):
GPU |
GPU Memory (GB) |
Precision |
GPU Count |
Profile Hash |
|---|---|---|---|---|
NVIDIA A100-SXM4-80GB |
80 |
BF16 |
1-2-4-8 |
|
NVIDIA B200 |
180 |
BF16 |
1-2-4-8 |
|
NVIDIA B300 (NVIDIA-B300-SXM6-AC) |
288 |
BF16 |
1-2-4-8 |
|
NVIDIA GB10 (DGX Spark) |
128 |
BF16 |
1 |
|
NVIDIA GB200 |
186 |
BF16 |
1-2-4-8 |
|
NVIDIA GB300 (Grace Blackwell Ultra) |
288 |
BF16 |
1-2-4-8 |
|
NVIDIA GH200 144GB |
144 |
BF16 |
1-2-4-8 |
|
NVIDIA H100 80GB HBM3 |
80 |
BF16 |
1-2-4-8 |
|
NVIDIA H200 |
141 |
BF16 |
1-2-4-8 |
|
NVIDIA L40S |
48 |
BF16 |
1-2-4-8 |
|
NVIDIA RTX 6000 Ada Generation |
48 |
BF16 |
1-2-4-8 |
|
NVIDIA RTX PRO 6000 Blackwell Workstation Edition |
96 |
BF16 |
1-2-4-8 |
|
Note
The GB10 features 128GB of LPDDR5x unified memory shared between CPU and GPU, providing seamless memory access across the Grace Blackwell Superchip architecture.
For optimal performance and stability, we recommend using one of the above tested configurations. While other NVIDIA GPUs with 48GB or more memory may work, they have not been officially tested and may exhibit degraded performance or stability issues.
Testing Locally Available Hardware#
Begin with Docker-supported operating system
Install Docker - minimum version: 23.0.1
Install NVIDIA Drivers - minimum version: 590.44.01 with CUDA 13.1
Install the NVIDIA Container Toolkit - minimum version: 1.13.5
Verify your container runtime supports NVIDIA GPUs by running
docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi
Example output:
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 590.44.01 Driver Version: 590.44.01 CUDA Version: 13.1 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA H100 80GB HBM3 Off | 00000000:01:00.0 Off | 0 |
| N/A 35C P0 45W / 700W | 0MiB / 81559MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
+-----------------------------------------------------------------------------+
Note
For more information on enumerating multi-GPU systems, refer to the NVIDIA Container Toolkit’s GPU Enumeration Docs