LinkX 100G-PAM4 Product Line Overview

The LinkX product line consists of direct attached copper (DAC), active copper cables (ACC), as well as both multimode and single-mode transceivers and crossover fibers.

• DAC consists of shielded copper wire with additional RFI foil shielding.

  • For 100G-PAM4 speeds, these can reach 3 meters. These have near zero latency and <0.1 Watts.

  • 30AWG is used for cables <2m and 26AWG up to 3m.

  • Available in 800G straight and 400G or 200G and QSFP112 and OSFP split ends.

• ACC is a DAC cable that includes pre-emphasis signal booster ICs (not a DSP), aka Linear ACCs.

  • Extends the reach to 3, 4, and 5-meters.

  • 1.5 Watts on 800G end and 0.6W and 0.35W for split ends, with very low latency.

  • 30AWG is used for cables <2m and 26AWG up to 3m.

  • Available in 800G straight and 400G or 200G and QSFP112 and OSFP split ends.

• Multimode optics convert electrical signals to drive 850nm laser light into 50-micron large diameter optical fibers.

  • Signals sent down the large diameter fiber take different paths (modes) to arrive at the end detector and become distorted as fiber length increases.

  • Multimode has a 50-meter maximum reach.

  • Large fiber 50-um diameter core is easy to align fibers with lasers and detectors resulting in lower costs to manufacture.

  • Fast and easy-to-align and manufacture minimizes costs compared to single mode optics.

  • 800G twin-port OSFP 17-Watt (max.) transceivers.

  • 400G QSFP112 or OSFP 9-Watt (max.) transceivers.

  • Accepts 50-meter straight or 1:2 splitters fiber cables.

  • 1:2 fiber splitters are used in both transceiver ports creating 4 links of 2x100G-PAM4 (200G).

  • 200Gb/s multimode transceivers are not offered and when used with 2-channel fiber split ends enables 200G and reduces power consumption.

• Single-mode optics main advantage is the long reach it provides using tiny 9-um diameter fiber core, which holds the signal together out to enormous lengths up to 2km.

  • The tiny core diameter is difficult to align and manufacture, so transceivers are more expensive than multimode transceivers.

  • 800G twin-port OSFP 17-Watt transceivers.

  • 400G QSFP112 or OSFP 9-Watt transceivers.

  • 200Gb/s multimode transceivers are not offered and when used with 2-channel fiber split ends enables 200G and reduces power consumption.

  • The main advantage of single-mode optics is the long reach it provides using:

    • Accepts 50-meter 1:2 splitter fibers for switch to adapter links.

    • Accepts straight fibers up at 100m, 500m, and 2km reaches for switch-to-switch applications. NVIDIA supplies up to 100m cables.

• Twin-port OSFP 800Gb/s finned top cables and transceivers are used only in switches.

• Twin-port OSFP 800Gb/s flat top cables and transceivers are available for use in DGX-H100s and NVLink4 switching systems in single-mode and multimode optics.

• The flat-top versions have the same 800G twin-port OSFP transceiver internals as with finned-top devices and use the riding heat sinks in liquid-cooled systems and large DGX H100 chassis for the GPU links.

• The 4-channel 400G transceivers use the OSFP or QSFP112 form-factor which has the same electronics and optics in both.

A series of DACs and active optical cables (AOCs) are offered for backwards compatibility with 2x200G twin-port OSFP on one end and 2x 200G or 2x 100G QSFP56/28 on split ends for linking to HDR/200GbE and 100GbE/HDR100 adapters and switches. These DACs and AOCs are the only cables that can link NDR/400GbE to HDR/200GbE systems. Transceivers are not available.

Additionally, QSFP112 cages in ConnectX-7 adapters and BlueField-3 DPUs are backwards compatible and support QSFP56 200G with 4x50G-PAM4 and QSFP28 100G with 4x 25G-NRZ. The cages accept the different connector types, and the cards can down shift to slower line rates.

100G-PAM4 multimode and single-mode transceivers cannot downshift to slower line rates of 50G-PAM4 and 25G-NRZ.

100G-PAM4 Series: Twin-port OSFP, OSFP, and QSFP112 Plugs

Electronics, optics, and copper wires are housed in metal shell plugs called form-factor plugs. The metal plugs have many code name extensions based on the single-channel; small-form-factor plug (SFP).

SFP can be proceeded by Q for quad or 4-channels (QSFP) and for 8-channels, quad -- double density (QSFP-DD) and octal (OSFP). NVIDA created an 8-channel transceiver called the twin-port OSFP that has 8 electrical channels and two optical 4-channel ports. Numbers at the end indicate the maximum Gb/s speed rating of the connector e.g., 28, 56, 112 for QSFP28, QSFP56, QSFP112. Also note that InfiniBand and Ethernet use slightly lower-speed ratings than the connector maximum speed, e.g. QSFP112 supports 100G line rates.

The 100G-PAM4 line uses only four connector types for DAC, ACC, AOC cables and transceivers:

Connectors used in ConnectX-7 and BlueField-3 have flat tops and use the riding cage heat sinks.

The MPO-12/APC is used in all the transceivers, except the 2xFR4 2km twin-port OSFP which uses LC optical connectors.

Twin-port OSFP, OSFP, and QSFP112 Connector Plugs

Twin-port 2x400G finned top OSFP	Twin-port 2x400G flat top OSFP	Single-port OSFP	Single-port QSFP112

Twin-port OSFP and OSFP Plugs

The octal small form-factor plug, or OSFP, has become the preferred form-factor for high-speed applications, such as artificial intelligence and HPC networking, as it offers future expansion with more channels, more space for components, and higher power dissipation capabilities. The twin-port OSFP 800G plug has 8-channels of electrical signaling for the switch and two 400Gb/s engines inside the transceiver that exit to two 400G optical or copper ports. Extra cooling fins are used on top to support 17-Watt transceivers, hence the name “2x400G twin-port OSFP finned-top.”

The 800G twin-port OSFP is also offered in a flat top version and a 400Gb/s single-port, flat-top OSFP is offered. These are all the same size, but the twin-port OSFP finned top version is taller due to the heat sink.

The three OSFP versions are:

• 800G finned-top, twin-port, 8-channel, 2x400G OSFP for Quantum-2 and Spectrum-4 SN5600 Ethernet air-cooled switches.

• 800G flat-top, twin-port, 8-channel, 2x400G OSFP for linking DGX H100 Cedar7 GPU links which use internal cage riding, air-cooled, heat sinks for liquid-cooled systems. This has the same internals as the finned top version.

• 400G flat-top, single-port, 4-channel, OSFP for ConnectX-7/OSFP network adapters using cage riding heat sinks.

800G twin-port OSFP finned-top connectors are also used to construct passive copper cables, active copper cables, and active optical cables using twin-port OSFP, OSFP, QSFP112, and QSFP56 connector ends. These parts are available in various combinations including 800G twin-port OSFP with flat tops instead of fins.

Lastly, a long reach single mode, 2km twin-port OSFP called 2xFR4 transceiver will be available at the end of 2023 that uses a finned-top but with a lid on top of the fins creating a closed channel for additional cooling.

Twin-port OSFP Open Finned Top Closed Finned Top

QSFP112 Plugs

QSFP112 form-factor is a single-port, 4-channel, 400G for ConnectX-7/QSFP112 and BlueField-3/QSFP112 DPUs.

• QSFP112 is without cooling fins on top and uses the cooling fins located on adapter and DPU connector cages.

• QSFP112 cannot be used in twin-port OSFP switch cages nor single-port ConnectX-7 adapters based on OSFP.

Twin-port OSFP, OSFP, and QSFP112 Connector Cages for Switches, ConnectX-7, and BlueField-3 DPUs

Two optical connectors are used for 100G-PAM4:

• MPO-12/APC 8-fiber: Multiple-Push-On,12 fiber, Angled Polished Connector for single-mode and multimode optics.

• LC duplex: 2-fiber Lucent Connector for 2km 2xFR4 transceivers for single-mode optics only.

The MPO-12/UPC, Ultra-flat Polish Connector, using 8-fibers is used for 50G-PAM4 backwards compatibility links for 200GbE, HDR, 100GbE, and EDR SR4 transceivers.

MPO-12/APC Optical Connectors

The MPO-12 optical connector is a ceramic block with holes that contain the ends of multiple optical fibers in either single-mode or multimode types. The ceramic blocks are made with different numbers of holes 8, 12, 16, 24, etc. but the 100G-PAM4 series uses 8-fibers but is labeled MPO-12.

Some of the light sent into a fiber reflects backwards from the fiber end face. Slower speed electronics and optics are less sensitive to back reflection created inside the optical fiber. Hence, 25G-NRZ (4x25G-NRZ SR4) and some 50G-PAM4 transceivers (4x50G-PAM4 SR4) use a different polish: MPO-12/UPC or Ultra-flat Polished Connector.

100G-PAM4 transceivers are more sensitive to back reflections and use the MPO-12/APC or Angled Polished Connector. This has an 8-degree polish on the end that causes the back reflections to be diverted into the fiber side cladding and away from the transmitter.

For 100G-PAM4 optics:

• MPO-12/APC is used for both single-mode and multimode optics.

• Two MPO-12/APC optical connectors are used with the twin-port 2x400G OSFP transceivers for two ports of 4-channels of 100G-PAM4 each.

• A single MPO-12/APC optical connector is used with 4-channel, 400G OSFP and QSFP112 transceivers.

• Only 8-fibers are used with 4 fibers for transmission and 4-fibers for receiving.

• NVIDIA supplied fiber cables have NVIDIA green plastic shells on the MPO ends to denote APC type.

• Transceivers have alignment pins and MPO-12/APC connectors have alignment holes. This is important to know when using trunk cables which may have pins or holes in the connector end. Inserting a pinned connector into a transceiver will damage both. A white dot or hole in the plastic side indicates fiber pin-1 side.

• The MPO-12/APC (green) is not compatible with the MPO-12/UPC (blue) ultra-flat polished connector used with 40G-200Gb/s transceivers with 50G-PAM4 and 25G-NRZ modulation as it has a flat polish.

• 400Gb/s QSFP-DD single mode DR4 transceivers used in Spectrum-3 SN4000 and Spectrum-4 SN5400 switches also use the 8-fiber MPO-12/APC. The DR4 uses 8-channels of 50G-PAM4 electrical and 4-channels 100G-PAM4 optical. This is the same optical connector used with twin-port and single-port 100G-PAM4 transceivers and can be used to link QSFP-DD with twin-port OSFP transceivers linking QSFP-DD 8x50G-PAM4 systems to 100G-PAM4 systems.

LC Duplex Optical Connectors

The 2-fiber Lucent Connector, or LC duplex, is typically used for single channel links using a transmit fiber and a receive fiber. This applies to multimode and single-mode optics.

Running parallel fibers over long reaches becomes expensive so transceivers use multiple lasers with different wavelengths which are combined or “multiplexed” into a single fiber for transmission and filtered back out at the receiver. One fiber for transmission and one for receiving but carrying 4 or 8 channels.

For long reaches up to 2-kilometers, two 2-fiber LC duplex optical connectors are used in the Far Reach 4-channel 2xFR4 twin-port OSFP transceiver. These are used to link 2x400Gb/s NDR InfiniBand and Spectrum-4 Ethernet switches together and linking clusters across campuses by using only two fibers in each optical connector.

MPO-12/APC, MPO-12/UPC, and Duplex LC Optical Connectors

MPO-12/APC		MPO-12/UPC	Duplex LC