LinkX User Guide for 400Gbps 100G-PAM4 OSFP & QSFP112-based Cables and Transceivers
LinkX User Guide for 400Gbps 100G-PAM4 OSFP & QSFP112-based Cables and Transceivers

Transceivers and Fiber Details: 100G-PAM4

Twin-port OSFP single-mode transceivers house two complete multimode or single-mode optical engines inside that exit to two, 4-channel MPO-12/APC optical connectors creating the twin-ports. Twin-port transceivers can be linked to each other forming an 800Gb/s link and can be linked to two or four 400G QSFP112 or OSFP transceivers using two straight or 1:2 splitter fiber cables forming either two 400G or four 200Gb/s links.

  • Twin-port OSFP transceivers can link to any combination of 2 or 4 OSFP and/or QSFP112 at the same time, as the internals are the same for both OSFP and QSFP112.

  • The fiber types (straight or splitter) must be the same in both twin-port transceiver ports as the switch and transceiver combination can only operate in either 2x400G or 4x200G split mode and cannot be configured with one straight and one 1:2 splitter fiber.

  • Twin-port 800G AOCs are not offered for 800G but are offered for 2x200G backwards compatibility to HDR and 200GbE.

Offered in multimode optics up to 50-meters and single-mode up to 100-meters and 500m and 2k-meters for long switch to switch links. The lengths chosen are related to not just the optics capabilities but also minimizing the overall link latencies including the switch buffer timing, optics latencies and capabilities, adapter timings. A 2km 2xFR4 version is also offered using two LC duplex optical connectors.

Transceiver reach and optics type nomenclature is as follows:

Twin-Port OSFP

  • 2x SR4: Short Reach 4-channel multimode optics; 800Gb/s; 50-meters maximum reach, aka SR8

  • 2x DR4: Datacenter Reach 8-channel; 800Gb/s; 100 and 500-meters maximum reach, aka DR8

  • 2x FR4: 2x Far Reach 4-channel; 800Gb/s; 2km maximum reach

Single-Port OSFP or QSFP112

  • SR4: Short Reach 4-channel multimode optics; 400Gb/s; 50-meters maximum reach

  • DR4: Datacenter Reach 4-channel single-mode optics; 400Gb/s; 100m and 500-meters maximum reach

  • FR4: Far Reach 4-channel, 400Gb/s, 2km maximum reach -- is not offered

2xSR4 and 2xDR4 are sometimes referred to in datasheets as 800G DR8 and 800G SR8. These are twin-port OSFP transceivers with 8-channels electrical (DR8) with two ports of 400G optical (2x400G).

The 400G InfiniBand and Spectrum-4 SN5600 Ethernet switches use exclusively the finned-top, twin-port, 2x400G OSFP connector. The twin-port OSFP employs 8 electrical pins all in one row, straight across for connection to the switch. Twin-port devices used in air-cooled switches have additional cooling fins on top of the connector as the transceivers dissipate 15-17 Watts each, and there are very few air-cooling holes in switch front panels.

Twin-port transceivers support two ports of 4x100G-PAM4 using two MPO-12/APC optical connectors. Each twin-port transceiver supports:

  • Two straight fiber cables with two 400G transceivers

  • Two 1:2 splitter cables supporting four transceiver ends. The 1:2 splitter fiber ends have 2 fiber sets (2-channels) and when used with 400Gb/s transceivers results in 200Gb/s rate.

  • 400G transceiver cannot be downshifted to 50G-PAM4 for 200G. 200G transceivers are not offered.

As most AI accelerated and HPC systems have a typical reach of less than 50-meters to limit the latency delay due to the light traveling in the fiber at 4.5 ns per meter, the 2xSR4 (aka SR8) is the most popular transceiver in addition to it being a less expensive than single mode optics. The 2xSR4 (aka SR8) consists of two VCSEL-based 850nm SR4 transceivers inside the twin-port OSFP shell. Two MPO-12/APC optical connectors exit the back of the transceiver supporting three configurations:

  • 800G-to-800G switch-to-switch using two 2xSR4s and two 4-channel fibers.

  • 800G-to-2x 400G switch-to-2x 400G adapter switch using a 2xSR4 and two 4-channel straight fibers to two 400G transceivers.

  • 800G-to-4x 200G switch-to-4x 200G adapters switch using four 2xSR4s and two 2-channel fibers splitters.

800G-to-2x 400G where each 400G port supports a 400G transceiver or be split by two creating a total of four 200Gb/s runs.
Both fibers must be the same straight or splitter and not one of each as the transceiver can only act as straight or split mode for both ports.

  • 50-meter transceiver design assumes two optical patch panels in the link budget.

  • Pull tab color: tan

  • Transceiver label: NVIDIA green

  • Reach marking: Blue stripe indicating multimode 50-meters

  • 17 Watts max.

2xSR4 800Gb/s Twin-port OSFP Finned and Flat Top, 50-meter, Multimode Transceivers

image2023-7-12_9-24-55.png

Single mode optics is all about long fiber reaches. For large system, the 2xDR4 single mode transceiver (aka DR8) offers long lengths and is typically used from 50-meters to 100-meters although supports upwards from 1-meter. The 2xDR4 consists of two transceiver engines using 1310nm externally modulated lasers (EML).

  • Transceiver design assumes two optical patch panels in the link budget.

  • Pull tab color: Yellow

  • Transceiver label: NVIDIA green

  • Reach marking: White indicating 100-meter single mode

  • 17 Watts max.

2xDR4 800Gb/s Twin-port OSFP Finned and Flat Top, 100-meter, Single-Mode Transceivers

image2023-7-12_9-26-17.png

A 500-meter twin-port OSFP single-mode transceiver is offered using two parallel 8-fiber MPO-12/APC optical connectors. This transceiver has the same design and specifications as the 100-meter version but tested to 500-meters and may use different error correction schemes.

  • Used to link two switches at 800Gb/s or three switches together at 400Gb/s

  • Crossover fibers cables longer than 100-meters are not offered by NVIDIA

  • Transceiver design assumes two optical patch panels in the link budget

  • Pull tab color: Yellow

  • Transceiver label: NVIDIA green

  • Reach marking: Purple indicating 500-meter single mode

  • 17 Watts max.

  • NVIDIA does not supply fibers longer than 100-meters

2xDR4 800Gb/s Twin-port OSFP Finned and Flat Top, 500-meter, Single-Mode Transceiver

image2023-7-12_9-29-1.png

A 2-kilometer twin-port OSFP single mode 2x Far Reach 4-channel transceiver (2xFR4) uses two 2-fiber LC optical connectors each carrying 400Gb/s multiplexed optical signals. These are used to link two Quantum-2 or Spectrum-4 OSFP switches together over long distances and complex networking infrastructures.

Four data signal channels, each using different laser wavelength light, are multiplexed into a single fiber for transmission and filtered out separately in the receiver. This saves costs associated with long fiber reaches, installation, and maintenance.

  • The fibers cannot be split into separate transceivers as with parallel fiber 2xDR4 transceiver designs.

  • Twin-port OSFP 2xFR4 100G-PAM4 cannot interface with a 4x50G-PAM4 FR4 transceiver due to different modulation schemes and the 2xFR4 cannot downshift to 50G-PAM4.

  • Used to link two switches at 800Gb/s or 3 switches at 400Gb/s.

  • Transceiver design assumes two optical patch panels in the link budget.

  • Pull tab color: Yellow

  • Transceiver label: NVIDIA green

  • Reach marking: Bright green indicating 2km single mode

  • 17 Watts max.

  • NVIDIA does not supply the LC-to-LC fiber cables, but they are based on standard optical industry practices and readily available from 3rd party suppliers.

Note: To link two transceivers directly together, one end should have the transmit fiber reversed with the receive fiber to align the transmit lasers with the receive photodetectors. Alternately, introduce and cross over fiber somewhere in the link.

Twin-port 2x400G OSFP 2xFR4 Transceiver and LC Optical Connector

image2023-7-12_9-30-52.png

  • The 2xFR4 transceiver uses a finned-top but with a lid on top of the fins creating a closed channel for additional cooling.

  • The closed finned top 2xFR4 transceivers can only be used in switch configurations with reverse air flow where the cooling air enters at the transceiver side of the chassis providing additional cooling.

    Twin-port OSFP Open Finned Top Closed Finned Top

    image2023-8-17_12-19-17.png

The single-port 4-channel transceivers are offered in both OSFP and QSFP112. QSFP112 was industry standardized much later than the OSFP, is significantly smaller, and fits on the restricted space of the BlueField-3 card with many components.

Single port, 4-channel, OSFP transceivers are used in network adapters only and connect to twin-port, 8-channel, OSFP transceivers inserted in switches.

The single-port OSFP is a flat-top, 4-channel version with cooling fins located on top of the adapter board connector cages called riding heat sinks (RHS). The twin-port OSFP and single-port OSFP are both physically the same size, but the single-port supports only one, 4-channel, MPO-12/APC optical connector and contains one, 400Gb/s, optical engine..

  • OSFP and QSFP112 have the same technical and internal specifications only with different connector metal shells.

  • 400G OSFP transceivers are only used in ConnectX-7/OSFP network adapters -– not used in BlueField-3/QSFP112 DPUs or Quantum-2, or Spectrum-4 switches.

  • 400G QSFP112 is used in ConnectX-7/QSFP112 and BlueField-3/QSFP112 -– not used in switches.

  • Available multimode up to 50-meters, and single-mode transceiver up to 100-meters.

  • Transceiver design assumes two optical patch panels in the link budget.

  • No QSA port adapter is offered for inserting QSFP112 into OSFP port adapter for use in OSFP cages.

  • 4-channel 100G-PAM4

  • MPO-12/APC 4-channel optical connector

  • Pull tab color: Tan

  • Transceiver label: NVIDIA green

  • Reach marking: Blue

  • 9 Watts max.

  • 4-channel 100G-PAM4

  • MPO-12/APC 4-channel optical connector

  • Pull tab color: Yellow

  • Transceiver label: NVIDIA green

  • Reach marking: White

  • 9 Watts max.

Multimode and Single-mode OSFP and QSFP112 Single-port Transceivers

image2023-7-12_9-32-24.png

  • 200Gb/s OSFP or QSFP112 transceivers (NDR200) are not offered as the 100G-PAM4 electronics cannot downshift to 50G-PAM4.

  • When 400Gb/s transceivers are used with 1:2 splitter fibers, only two channels are activated creating a 200Gb/s transceiver.

  • Power consumption drops from 8 Watts multimode to 5.5 Watts; single mode from 9 Watts to 6.5 Watts.

Two Splitter Cables Create 200G Transceivers from 400G Transceivers

image2023-7-12_9-33-56.png

NVIDIA transceivers have an NVIDIA green label on the connector body:

  • 50-meter multimode transceivers have a blue stripe on the connector body and tan pull tabs.

  • 100-meter single-mode transceivers have a white stripe and yellow pull tabs.

  • 500-meter single-mode transceivers have a purple stripe and yellow pull tabs.

  • 2k-meter single-mode transceivers have a bright green stripe and yellow pull tabs.

Multimode fiber: aqua colored cable jacket
Single-mode fiber: yellow cable jacket
MPO-12/APC optical connector: green shell
MPO-12/UPC optical connector: blue shell

Cable and Transceiver Labeling and Colors

image2023-7-12_9-35-46.png

image2023-7-12_9-36-14.png

image2023-7-12_9-36-34.png

© Copyright 2023, NVIDIA. Last updated on Aug 29, 2023.