NVIDIA Mission Control North-South Network GB200/GB300 Deployment Guide - External

Contents:

• Introduction
  • Reference Documentation
  • Downloads
  • Ethernet Switch Hardware
  • Compute Tray Network Ports
    • GB200 Compute Tray
    • GB300 Compute Tray and NVLink Switch
• Networking Planning and Design
  • North-South Ethernet Networking
    • IP subnet range/CIDR requirements as defined by BCM network naming conventions. Applicable to both GB200 and GB300.
    • GB300 Subnetting Breakout
    • Ethernet Planning
• Configure Cluster Ethernet Networking
  • Summary
  • USB Preparation
  • GB200/GB300 Rack Inventories Workflow
  • Manual Process
    • Stage1: Preparation: Build IP breakout
    • Stage 2: ON Field Deployment
• Ethernet Cluster Networking Validation
  • Fabric Underlay Routing
    • Platform Health Check
    • Cabling Quality (L1)
    • Fabric Underlay Routing
    • Customer Network Reachability
    • Negative Testing
• NVLink Switch Planning and Design
  • Confirm Accurate Hardware Models and Quantities
  • Network and Connectivity Requirements
    • Kubernetes Admin and User nodes to BTOR switch connectivity
    • NVLink Switch to OOB Switch connectivity
    • Routable IP Address Allocation
  • NVLink Switch Design
  • NVLink Switch Integration with BCM Verification
  • NVLink Switch Tray Check and Configuration
    • NVLink Switch Workflow Diagram:
• Appendix: Configuration Examples
  • Section 1: OOB (example)
    • Section: A1: IPMI Configuration (OOB)
  • Section 2: TOR (example)
    • BTOR-0[1-2] Basic (Border Leaf)
    • BTOR-0[x-y] CONTROLNODE BONDING
    • BTOR-01 SVI
    • BTOR-02 SVI
    • BTOR-0[1-2] EVPN/VXLAN/RoCE
    • BTOR-0[1-2] BGP VRF
    • BTOR-0[1-2] BGP EDGE
    • BTOR-0[1-2] POLICY
    • BTOR-0[1-2] to SPINEs
    • BTOR-0[1-2] DHCP-Relay
    • BTOR-0[1-2] STORAGE VRF
    • BTOR-0[1-2] POLICY 2
    • Inband Policy Inserts Default Route
    • Connection Towards Customer
    • Aggregate Summary Routes towards Edge
  • Section 3: GB200/GB300 Rack Inventory (Example)