Introduction#

This document details deploying the North-South Networking for a DGX GB200 NVL72 SuperPod.

You must complete the physical installation of the network switches (SN5600 and SN2201) before you deploy the switches, either manually, or from within the tool included in BCM 11. In addition, you can find information about the intended deployment in a site information blueprint (SIB).

Overall, the DGX SuperPOD Deployment Guides contain essential information and should be consulted before deploying the configuration:

Reference Documentation#

Reference Document

Description

Ethernet Architecture Design GB200: Logical/Physical (Internal)

DGX GB200 SuperPOD Logical/Physical Reference Design

BCM11 Network Bringup Guide (Internal)

Network BringUP Deployment Guide

Hardware Compatibility List

Physical Cable / Optics / Transceiver compatibility

Downloads#

Download File

Description

NVLink Switch (NVOS)

NVOS (Downloads > Switches & Gateways > Switch Software > NVOS_NVLINK4)

Cumulus OS 5.11.0

CumulusOS (Downloads > Switches & Gateways > Switch Software > Nvidia Cumulus Switch)

InfiniBand OS

InfiniBand (Downloads > Switches & Gateways > Switch Software > Nvidia Onyx)

Physical Hardware#

Network Switches:

Switch Model

Description

SN5600

(INBAND) Cumulus OS, 64 ports of 800Gb

SN2201

(OOB) 48 1G Base-T