NDR Overview

The introduction of 400 Gbps (NDR) InfiniBand doubles network performance compared to HDR, and the increase from HDR’s 40 switch ports to 64 ports greatly reduces the amount of equipment needed to implement a customer fabric. NDR also offers excellent backwards compatibility with existing HDR and EDR deployments.

This chapter assumes familiarity with InfiniBand concepts covered in InfiniBand Cables Primer.

  • Compared to HDR, NDR introduces several new capabilities:

  • Link speeds of 400 Gbps (four lanes of 100 Gbps) and 200 Gbps (two lanes of 100 Gbps). The latter is referred to as NDR200 and differs from HDR, which is four lanes of 50 Gbps.

  • Octal Small Form-factor Pluggable (OSFP) connectors, which enable higher port densities than QSFP by supporting two InfiniBand links per physical connector.

  • Twin cables and transceivers—pairs of cables or transceivers that share an OSFP connector.

  • QSFP112 connectors, which are used on DPUs where space is at a premium.

Due to higher speed and increased power requirements, the mix of NDR cables and transceivers has changed from HDR:

  • The reach of DAC cables, which are the least expensive and lowest power cables, is shorter than for HDR. A range of NDR ACC cables provides an intermediate step in distance and power draw between copper and optical connectivity.

  • Active optical cable (AOC) technology, in which the transceivers and fiber are a single assembly, is not practical for NDR. Distances greater than a few meters are managed by NDR optical transceivers and passive multi-strand MPO fiber cables.

  • A new family of MPO cables, called MPO/APC (Angled Physical Contact, or Angle-Polished Connector), increases link reliability and distance. These cables are not compatible with HDR or EDR transceivers.

  • Splitting NDR optical links into two NDR200 links introduce the need for MPO fiber splitters.

  • Backwards compatibility with HDR, HDR 100, and EDR involves a new family of cables having OSFP connectors at the switch end.

  • The OSFP twin-port approach somewhat constrains the usage of adjacent ports. For example, cables from the InfiniBand port pair must be the same media type.