NVIDIA HGX AI Factory

NVIDIA Enterprise Reference Architecture

• Abstract
• Overview of the Reference Architecture (RA)
• Components
  • NVIDIA HGX H100/H200 8-GPU Baseboard
  • NVIDIA HGX B200 8-GPU Baseboard
  • HGX H100, H200 or B200 8-GPU Systems
  • Control Plane/Management Nodes
  • HGX H100, H200 or B200 8-GPU System Networking
  • Compute (Node East/West) Ethernet Networking
  • Converged (Node North/South) Ethernet Networking
• Networking Hardware
  • NVIDIA BlueField-3 B3140H SuperNIC
  • NVIDIA BlueField-3 B3220 DPU
  • NVIDIA ConnectX-7 SmartNIC (variant)
  • NVIDIA ConnectX-8 C8240 SuperNIC (variant)
  • NVIDIA SN5600/SN5610 Switch
  • NVIDIA SN2201 Switch
• Networking Physical Topologies
  • Compute (Node East/West) Network
  • Compute Fabric Excluded for Pure Inference Deployments
  • Converged (Node North/South) Network
  • Out-of-Band Management (Node) Network
• Network Logical Architecture
  • Enterprise RA Scalable Unit (SU)
  • Spine-Leaf Networking
  • Out-of-Band (OOB) Management Network
  • Optimized Use Case
  • Architecture Details
    • Compute (Node East-West) Fabric Table
    • Converged (Node North-South) Fabric Table
  • 32 Nodes with 256 HGX H100, H200 or B200 SXM GPUs
  • 64 Nodes with 512 HGX H100, H200 or B200 SXM GPUs
  • 128 Nodes with 1024 HGX H100, H200 or B200 SXM GPUs
• Software
• NVIDIA-Certified Storage
• Services
• Summary

Appendix

• Node Configurations
  • Switches and Cables

Notices

• Notices
  • Notice