Appendix D#

NVIDIA Enterprise Reference Architecture: NVIDIA H200 NVL or RTX PRO™ 6000 Blackwell Server Edition and NVIDIA Spectrum-X Networking Platform#

This NVIDIA Enterprise Reference Architecture (Enterprise RA) is optimized for multi-node AI and hybrid applications, utilizing a 2-8-5-200 node architecture with either H200 NVL or RTX PRO 6000 Blackwell Server Edition GPUs. This setup is further enhanced by the NVIDIA Spectrum-X Networking Platform, which provides advanced networking capabilities tailored for AI workloads.

The Enterprise RA is a modular architecture based on an NVIDIA-Certified system, each with eight H200 NVL or RTX PRO GPUs. Using a four-node scalable unit (SU), this Enterprise RA scales up to 32 NVIDIA-Certified systems for a total of 256 H200 NVL or RTX PRO GPUs.

Use Cases#

  • AI Inference: Large (per node) and medium (per GPU) model parameter inference workloads

  • AI Training: Large to small model training and fine-tuning based on cluster sizing

NVIDIA H200 NVL Systems#

NVIDIA H200 NVL is ideal for lower-power, air-cooled enterprise rack designs, delivering acceleration for every AI and HPC workload regardless of size. With up to four GPUs connected by NVIDIA NVLink™ and a 1.5X memory increase, large language model (LLM) inference can be accelerated up to 1.7X and HPC applications achieve up to 1.3X more performance over the H100 NVL. With four H200 NVL GPUs interconnected with NVLink 2 or 4 bridges, the combined H200 GPU memory is 564 GB for unprecedented acceleration.

Figure 9. Example of a NVIDIA H200 NVL 8 GPU system configuration.

_images/ra-overiew-09.png

NVIDIA RTX PRO 6000 Blackwell Server Edition Systems#

The NVIDIA RTX PRO 6000 Blackwell Server Edition GPU is a high-performance GPU designed for server environments. It features 96GB of GDDR7 memory per GPU, significantly surpassing the 48GB GDDR6 memory of its counterpart, the NVIDIA L40S. This results in a substantial increase in memory bandwidth, reaching up to 1.6TB/s per GPU, compared to 864GB/s for the L40S. When configured in an 8-GPU node, the RTX PRO 6000 offers 768GB of GDDR7 memory and achieves an aggregate memory bandwidth of up to12.8TB/s, doubling the capacity and bandwidth of the L40S setup. This makes it particularly suited for demanding applications requiring high memory and bandwidth capabilities.

Figure 10. Example of a NVIDIA RTX PRO 6000 Blackwell Server Edition 8 GPU system configuration.

_images/ra-overiew-10.png