You can download a PDF of the full document here .
Introduction
Guide to Configuring 100G-PAM4 LinkX Interconnects for Quantum-2 InfiniBand and Spectrum-4 Ethernet OSFP-based Switches with ConnectX-7 Adapters and BlueField-3 DPUs based on twin-port OSFP, OSFP, and QSFP112 Cables and Transceivers
NVIDIA® LinkX® 400Gb/s Cable and Transceiver User Guides provides detailed information, figures, and ordering part numbers to assist in configuring cables and transceivers for use with network switches, BlueField® DPUs, and ConnectX® network adapters for both Ethernet and InfiniBand protocols. This document covers only cables and transceivers based on 100G-PAM4 modulation and a few specific parts for backwards compatibility linking to 50G-PAM4 devices.
Interconnect Documents
This guide is designed to be used in conjunction with additional documentation located in docs.nvidia.com/networking > Networking Interconnect. Click on configuration map to see information in this document in an easy-to-read presentation. This site is where the following LinkX cables and transceivers documents are provided.
400Gb/s Rate
The 400Gb/s rate cables and transceivers are used for both InfiniBand NDR Quantum-2 QM9700 and 400GbE SN56x0 Spectrum™-4 Ethernet OSFP-based, air-cooled switches. Additionally, the NDR InfiniBand parts can be used with the new XDR InfiniBand switches based on 200G-PAM4 and the ConnectX-8 network adapter. The 400Gb/s rate consists of 4-channels of 100G-PAM4 signaling and based on the octal small form-factor plug (OSFP) and the quad small formfactor plug (QSFP112).
New industry plug, cooling terminology has been added.
“Finned-top” transceivers are now called “Integrated Heat Sink” (IHS) as the cooling fins are integrated into the OSFP connector metal top. Used only in air-cooled switches.
“Flat-top” transceivers are now called “Riding Heat Sink” (RHS) as the cooling fins ride on top of the network card or systems internal cages. This is used for both air-cooled and liquid-cooled applications for DGX, ConnectX-7, and BlueField-3 DPUs.
For switches: two 400Gb/s ports are included in one 2x400G OSFP air-cooled switch cage in the switches and called a twin-port IHS (finned top) OSFP. Since there are two 400G engines in the twin-port OSFP transceiver, this has an 800Gb/s aggregate electrical data rate and may also be referred to as 2x400G optical.
For PCIe network adapters: ConnectX-7 and ConnectX-8 network adapters and BlueField-3 Data Processing Units (DPUs) use 4-channel, 400Gb/s single-port RHS (flat top) OSFP or QSFP112 cages with finned heat sinks on top of the board cages (RHS). The twin-port OSFP IHS devices cannot be used with network adapters and DPUs as the IHS finned top is considerably taller than the single port cage opening used on the cards.
LinkX cables and transceivers tie the switches and network adapters all together with copper cables and optical transceivers.
The 400Gb/s Spectrum-3 SN4700 and Spectrum-4 SN5400 Ethernet switches are constructed from 8-channels of 50G-PAM4 signal modulation and use the quad small form-factor plug with double density (QSFP-DD)—covered in another document. QSFP-DD devices are not used with InfiniBand.
Parts are Described by Use in the Switch
The cable part number descriptors are based on the connector used in the switches, as the devices used in network adapters may consist of multiple connector types such as the single-port OSFP, QSFP112, QSFP56, etc. These plugs or “form-factors” are used to contain optical transceivers and copper cables to form the switching networks linking CPU/GPU compute servers with storage subsystems and to other system clusters in the network. For example, an 800G twin-port OSFP-to-4x 200G QSFP112 splitter copper cable below is listed in the parts lists as 800Gb/s twin-port OSFP, not 200Gb/s QSFP112.
Cable Descriptions Denoted by the Switch-side Connector
