Interrupted Rack Layouts#
This documentation is part of NVIDIA DGX SuperPOD: Data Center Design Featuring NVIDIA DGX H100 Systems.
This section details some options for addressing rack layout challenges caused by data center architectural features. In all cases, overhead ladders or trays must span the racks on either side of the obstruction to facilitate cabling across the gap.
Use this documentation to learn about the following:
Overcoming Obstructions to Contiguous Racks#
Figure 2 shows a top view of how a site-specific obstruction, in this case a structural column within a row, can be accommodated.
Figure 2. Obstructions to contiguous DGX SuperPOD deployments#
Note
In this example, the tray has been located closer to the rear of the rack to clear the obstruction. If the obstruction was toward the rear of the rack, it might have been necessary to move the cable tray toward the front of the racks. This is not ideal but preferred over bending the tray around the obstacle. An aisle containment partition takes the place of the missing rack, to ensure that hot aisle/cold aisle integrity is maintained.
Spanning Deployments Across Two Rows of Racks#
Figure 3 shows how the standard layout can be modified to place racks in two rows. When spanning more than one row, cable length is an important factor. The total distance of travel must not exceed 50m for any InfiniBand multimode optical cable that connects a DGX H100 system to a leaf switch, and for optimum system performance, cable lengths should be limited to 30m.
Figure 3. Spanning across two rows of racks (top view of racks)#
The maximum distance for switch to switch interconnectivity using single mode optical cable is 500m. These distances cannot be extended using intermediary repeaters or switches. For overhead or underfloor cable trays that provide more than one bridge to adjacent rows, cables should be routed to the bridge that provides the shortest path to the destination rack.