NVIDIA CS8500 800 Port HDR Switch System User Manual (EOL)
NVIDIA CS8500 800 Port HDR Switch System User Manual

Port Connector Interfaces

The connector side of the switch has 20 leaf modules and each leaf module has 40 QSFP28 ports. The ports on each leaf module are placed in two rows, 20 ports to a row. The ports are labeled as shown below. The bottom row ports are flipped from the top row.

image2019-1-17_18-25-28.png

Port Numbering

image2019-1-17_18-29-39.png

Top and Bottom Ports

image2018-12-2_11-41-53.png

InfiniBand QSFP Connector Pin-out

Connector Pin Number

Connector Pin Name

Signal Description

1

GND

Ground

2

Tx2n

Transmitter Inverted Data Input

3

Tx2p

Transmitter Non-Inverted Data Input

4

GND

Ground

5

Tx4n

Transmitter Inverted Data Input

6

Tx4p

Transmitter Non-Inverted Data Input

7

GND

Ground

8

ModSelL

Module Select

9

ResetL

Module Reset

10

Vcc Rx

+3.3 V Power supply receiver

11

SCL

2-wire serial interface clock

12

SDA

2-wire serial interface data

13

GND

Ground

14

Rx3p

Receiver Non-Inverted Data Output

15

Rx3n

Receiver Inverted Data Output

16

GND

Ground

17

Rx1p

Receiver Non-Inverted Data Output

18

Rx1n

Receiver Inverted Data Output

19

GND

Ground

20

GND

Ground

21

Rx2n

Receiver Inverted Data Output 3

22

Rx2p

Receiver Non-Inverted Data Output 3

23

GND

Ground

24

Rx4n

Receiver Inverted Data Output 3

25

Rx4p

Receiver Non-Inverted Data Output 3

26

GND

Ground

27

ModPrsL

Module Present

28

IntL

Interrupt

29

Vcc Tx

+3.3 V Power supply transmitter

30

Vcc 1

+3.3 V Power Supply

31

LPMode

Low Power Mode

32

GND

Ground

33

Tx3p

Transmitter Non-Inverted Data Input

34

Tx3n

Transmitter Inverted Data Input

35

GND

Ground

36

Tx1p

Transmitter Non-Inverted Data Input

37

Tx1n

Transmitter Inverted Data Input

38

GND

Ground

InfiniBand QSFP Connector Symbol

image2018-12-2_11-59-24.png

QSFP Connector Male and Female Views

image2018-12-2_11-59-56.png

QSFP Cable Power Budget Classification

This switch systems is designed for active cable usage with a max power of 4.9W per module. This is power level 6 according to the QSFP Public Specification.

Approximately 10% of switch power is dissipated through the air. Switch comes with the air flow pattern of air entering through the spine side and exiting through the connector side. There are few fans within the system to manage air cooling:

  • Each PSU is having its own built in fan, this fan is under PSU control and monitoring. Its speed and health are monitored but can’t be altered.

  • MNG FRU is having 3 fans (2+1 redundant) at its front panel to take care of its own cooling.

  • MNG FRU is also having 2 additional fans (1 + 1 redundant) for chassis air cooling. Those fans are providing minimum airflow within the chassis to avoid air stagnation and system thermal runaway.

Warning

If there are plans to use system with not all FRUs installed (MNG, Spine or Leaf) the user must install the relevant blank unit into the unused slot/s

90% of the switch system's power is dissipated through a water cooling device. The switch's inner water loop is sourced by either the CDU or the AHX (depending on user’s facilities). There is a ball valve connector on incoming and outcoming hoses between the CDU and the switch to allow a leak-free connection and disconnection between the units. Also, Leaf and Spine FRUs are equipped with dripless quick-connect connectors to allow hot swapping operation. Connecting and disconnecting FRUs can be done under water pressure, there will be no leak from either the manifold nor the FRU.

Warning

Do not disconnect any other water connectors, it will cause immediate water flooding and subsequent system thermal shutdown.

Technology loop water (The switch's inner water loop) is filled with ultra-pure water with inhibitors and biocides to maintain long term reliable cooling. This water solution requires periodic maintenance – see the CS8500 Cooling System Support and Maintenance user manual for further details.

© Copyright 2023, NVIDIA. Last updated on May 22, 2023.