NVIDIA ConnectX-7 Adapter Cards Firmware Release Notes v28.36.2020
NVIDIA ConnectX-5 Adapter Cards Firmware Release Notes v16.35.3502 LTS

Firmware Compatible Products

These are the release notes for the NVIDIA® ConnectX®-7 adapters firmware Rev 28.36.2020. This firmware supports the following protocols:

  • InfiniBand - NDR200, NDR

  • Ethernet - 1GbE,10GbE, 25GbE, 40GbE, 50GbE1, 100GbE1, 200GbE2

  • PCI Express 5.0, supporting backwards compatibility for v4.0, v3.0, v2.0 and v1.1

1. Speed that supports both NRZ and PAM4 modes in Force mode and Auto-Negotiation mode.

2. Speed that supports PAM4 mode only.

Warning

When connecting an NVIDIA-to-NVIDIA adapter card in ETH PAM4 speeds, Auto-Neg should always be enabled.

NVIDIA SKU

Legacy OPN

PSID

Device Description

FlexBoot

UEFI x86

UEFI Arm

Enable/ disable exprom Feature

900-9X720-00E0-S0B /
900-9X720-00E0-S00

MCX750500B-0D0K /
MCX750500B-0D00

MT_0000000891

NVIDIA adapter card with four ConnectX-7; each up to 400Gb/s IB (default mode) or 400GbE; PCIe 5.0 x32; PCIe switch; crypto disabled; secure boot enabled

Present (Enabled)

Present (Enabled)

Present (Enabled)

Exists

930-9O000-0000-060

MCX755206AS-NEAT-N

MT_0000000892

NVIDIA ConnectX-7 VPI adapter card; 400Gb/s IB and 200GbE; dual-port QSFP; PCIe 5.0 x16 with x16 PCIe extension option; dual slot; secure boot; no crypto; tall bracket for Nvidia DGX storage

Present (Enabled)

Present (Enabled)

Present (Enabled)

Exists

The following are the drivers’ software, tools, switch/HCA firmware versions tested that you can upgrade from or downgrade to when using this firmware version:

Supported Version

ConnectX-7 Firmware

28.36.2020 / 28.36.1010 / 28.35.2000

MLNX_OFED

5.9-0.5.6.0.107 / 5.9-0.5.6.0 / 5.8- 1.1.2.1

Note: For the list of the supported Operating Systems, please refer to the driver's Release Notes.

MLNX_EN (MLNX_OFED based code)

5.9-0.5.6.0 / 5.8- 1.1.2.1/ 5.8- 1.0.1.1

Note: For the list of the supported Operating Systems, please refer to the driver's Release Notes.

WinOF-2

3.20.50010 / 3.10.51000 / 3.10.50000

Note: For the list of the supported Operating Systems, please refer to the driver's Release Notes.

MFT

4.23.0 / 4.22.1 / 4.22.0

Note: For the list of the supported Operating Systems, please refer to the driver's Release Notes.

mstflint

4.23.0 / 4.22.1 / 4.22.0

Note: For the list of the supported Operating Systems, please refer to the driver's Release Notes.

FlexBoot

3.6.901

Note: Please be aware that not all firmware binaries contain FlexBoot or UEFI, support may vary between cards.

UEFI

14.29.14

Note: Please be aware that not all firmware binaries contain FlexBoot or UEFI, support may vary between cards.

MLNX-OS

3.9.0900 onwards

NVIDIA Quantum-2 Firmware

31.2010.1404 onwards

Warning

NVIDIA does not support InfiniBand cables or modules not qualified or approved by NVIDIA.

Switch and HCAs InfiniBand Cable Connectivity Matrix

NVIDIA Quantum™ based switches and NVIDIA® ConnectX® HCAs support NDR (PAM4, 100Gb/s per lane), HDR (PAM4, 50Gb/s per lane) and EDR (NRZ, 25Gb/s per lane) technologies. As the ConnectX adapter cards are identified by their maximum supported throughput (e.g., ConnectX-6 VPI 100Gb/s card can support either 2-lanes of 50Gb/s or 4-lanes of 25Gb/s), the exact connectivity will be determined by the cable that is being used or by manual configuration.

As a reference:

Speed Mode

Speed Supported

Number of Lanes Used

NDR

400Gb/s InfiniBand

4 lanes of 100Gb/s

NDR200

200Gb/s InfiniBand

2 lanes of 100Gb/s

The following tables present the connectivity matrix, between NVIDIA Quantum based switches, ConnectX HCA, and the cables.

Switch-to-HCA Connectivity Matrix

Switch

Adapter

Cable

NDR Transceiver

NDR ACC/DAC

HDR AOC

HDR DAC

HDR100 DAC/AOC

EDR DAC/AOC

NVIDIA Quantum-2

NDR Switch

ConnectX-7 400Gb/s

2 × NDR

2 × NDR

N/A

N/A

N/A

N/A

NVIDIA Quantum-2

ConnectX-7 200Gb/s

4 × NDR200

4 × NDR200

N/A

N/A

N/A

N/A

VPI Protocol Support

ConnectX-7 supports having one port as InfiniBand and the second port as either InfiniBand or Ethernet.

To set the right configuration, run:

Copy
Copied!
            

mlxconfig -d <mst device> s LINK_TYPE_P1=1/2 LINK_TYPE_P2=1/2

where:

  • LINK_TYPE_P2 - sets the configuring protocol for port 2

  • (1/2) - values used for the different protocols:

    • 1 – for InfiniBand

    • 2 - for Ethernet

Validated and Supported NDR Cables

Speed

OPN

Description

NDR

MCP7Y00-N01A-FLT

NVIDIA passive copper splitter cable, IB twin port NDR 800Gb/s to 2x400Gb/s, OSFP to 2xOSFP,1.5m, flat top

NDR

MMS4X00-NL-FLT

NVIDIA twin port transceiver, 800Gbps,2xNDR, OSFP, 2xMPO12 APC, 1310nm SMF, up to 30m, flat top

NDR

MMS4X00-NS-FLT

NVIDIA twin port transceiver, 800Gbps,2xNDR, OSFP, 2xMPO12 APC, 1310nm SMF, up to 100m, flat top

NDR

MMA4Z00-NS-FLT

NVIDIA twin port transceiver, 800Gbps,2xNDR, OSFP, 2xMPO12 APC, 850nm MMF, up to 50m, flat top

NDR

MMA1Z00-NS400

NVIDIA single port transceiver, 400Gbps,NDR, QSFP112, MPO12 APC, 850nm MMF, up to 50m, flat top

NDR

MCA7J65-N001

NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 2x400Gb/s, OSFP to 2xQSFP112, 1m

NDR

MCA7J65-N002

NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 2x400Gb/s, OSFP to 2xQSFP112, 2m

NDR

MCA7J65-N003

NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 2x400Gb/s, OSFP to 2xQSFP112, 3m

NDR

MCA7J65-N004

NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 2x400Gb/s, OSFP to 2xQSFP112, 4m

NDR

MCA7J65-N005

NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 2x400Gb/s, OSFP to 2xQSFP112, 5m

NDR

MCA7J75-N001

NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 4x200Gb/s, OSFP to 4xQSFP112, 1m

NDR

MCA7J75-N002

NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 4x200Gb/s, OSFP to 4xQSFP112, 2m

NDR

MCA7J75-N003

NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 4x200Gb/s, OSFP to 4xQSFP112, 3m

NDR

MCA7J75-N004

NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 4x200Gb/s, OSFP to 4xQSFP112, 4m

NDR

MCA7J75-N005

NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 4x200Gb/s, OSFP to 4xQSFP112, 5m

NDR

MCA4J80-N003-FTF

NVIDIA Active copper cable, IB twin port NDR, up to 800Gb/s, OSFP, 3m, flat to finned

NDR

MCA4J80-N003-FLT

NVIDIA Active copper cable, IB twin port NDR, up to 800Gb/s, OSFP, 3m, flat to flat

Validated and Supported HDR Cables

Speed

OPN

Description

HDR

MFS1S00-HXXXE

Mellanox active fiber cable, IB HDR, up to 200Gb/s, QSFP56, LSZH, black pulltab, Xm

HDR

MFA7U10-H003

NVIDIA AOC splitter, IB twin port HDR, 400Gb/s to 2x200Gb/s, OSFP to 2xQSFP56, 3m

HDR

MFA7U10-H005

NVIDIA AOC splitter, IB twin port HDR, 400Gb/s to 2x200Gb/s, OSFP to 2xQSFP56, 5m

HDR

MFA7U10-H010

NVIDIA AOC splitter, IB twin port HDR, 400Gb/s to 2x200Gb/s, OSFP to 2xQSFP56, 10m

HDR

MFA7U10-H015

NVIDIA AOC splitter, IB twin port HDR, 400Gb/s to 2x200Gb/s, OSFP to 2xQSFP56, 15m

HDR

MFA7U10-H020

NVIDIA AOC splitter, IB twin port HDR, 400Gb/s to 2x200Gb/s, OSFP to 2xQSFP56, 20m

HDR

MFA7U10-H030

NVIDIA AOC splitter, IB twin port HDR, 400Gb/s to 2x200Gb/s, OSFP to 2xQSFP56, 30m

HDR

MFA7U10-H050

NVIDIA AOC splitter, IB twin port HDR, 400Gb/s to 2x200Gb/s, OSFP to 2xQSFP56, 50m

Tested NDR Switches

Speed

Switch Silicon

OPN # / Name

Description

Vendor

NDR

NVIDIA Quantum-2

MQM9790

NVIDIA Quantum-2 based NDR InfiniBand EVB Switch, 64 NDR ports, 32 OSFP ports, non-blocking switching capacity of 51.2Tbps, 2 Power Supplies (AC), Standard depth, Unmanaged, P2C airflow, Rail Kit, RoHS6

NVIDIA

NDR

NVIDIA Quantum-2

MQM9700

NVIDIA Quantum 2 based NDR InfiniBand Switch, 64 NDR ports, 32 OSFP ports, 2 Power Supplies (AC), Standard depth, Managed, P2C airflow, Rail Kit

NVIDIA

Tested 400GbE Switches

Speed

Switch Silicon

OPN # / Name

Description

Vendor

400GbE

Spectrum-3

MSN4410

24 QSFP-DD28 and 8 QSFP-DD ports, 400GbE 1U Open Ethernet Switch with Onyx

NVIDIA

400GbE

Spectrum-3

MSN4700

32 QSFPDD ports, 400GbE 1U Open Ethernet Switch with Onyx

NVIDIA

400GbE

N/A

Wedge 400

Wedge 400-48X 400GbE Data Center Switch

Facebook

400GbE

N/A

Cisco Nexus 3432D-S

Cisco Nexus 3432D-S, 32 fixed 400-Gigabit Ethernet QSFP-DD ports with backward compatibility for QSFP56, QSFP28, and QSFP+

Cisco

Tested 200GbE Switches

Speed

Switch Silicon

OPN # / Name

Description

Vendor

200GbE

Spectrum-3

MSN4600V-XXXX

64 QSFP56 ports, 200GbE 2U Open Ethernet Switch with Onyx

NVIDIA

© Copyright 2023, NVIDIA. Last updated on May 23, 2023.