These are the release notes for the NVIDIA® ConnectX®-7 adapters firmware Rev 28.36.2024. This firmware supports the following protocols:
InfiniBand - NDR200, NDR
Ethernet - 1GbE,10GbE, 25GbE, 40GbE, 50GbE1, 100GbE1, 200GbE2
PCI Express 5.0, supporting backwards compatibility for v4.0, v3.0, v2.0 and v1.1
1. Speed that supports both NRZ and PAM4 modes in Force mode and Auto-Negotiation mode.
2. Speed that supports PAM4 mode only.
When connecting an NVIDIA-to-NVIDIA adapter card in ETH PAM4 speeds, Auto-Neg should always be enabled.
NVIDIA SKU | Legacy OPN | PSID | Device Description | FlexBoot | UEFI x86 | UEFI Arm | Enable/ disable exprom Feature |
900-9X720-00E0-S0B / | MCX750500B-0D0K / | MT_0000000891 | NVIDIA adapter card with four ConnectX-7; each up to 400Gb/s IB (default mode) or 400GbE; PCIe 5.0 x32; PCIe switch; crypto disabled; secure boot enabled | Present (Enabled) | Present (Enabled) | Present (Enabled) | Exists |
930-9O000-0000-060 | MCX755206AS-NEAT-N | MT_0000000892 | NVIDIA ConnectX-7 VPI adapter card; 400Gb/s IB and 200GbE; dual-port QSFP; PCIe 5.0 x16 with x16 PCIe extension option; dual slot; secure boot; no crypto; tall bracket for Nvidia DGX storage | Present (Enabled) | Present (Enabled) | Present (Enabled) | Exists |
The following are the drivers’ software, tools, switch/HCA firmware versions tested that you can upgrade from or downgrade to when using this firmware version:
Supported Version | |
ConnectX-7 Firmware | 28.36.2024 / 28.36.2020 |
MLNX_OFED | 5.9-0.5.6.0.113 / 5.9-0.5.6.0.107 Note: For the list of the supported Operating Systems, please refer to the driver’s Release Notes. |
MLNX_EN (MLNX_OFED based code) | 5.9-0.5.6.0 / 5.8- 1.1.2.1 Note: For the list of the supported Operating Systems, please refer to the driver’s Release Notes. |
WinOF-2 | 3.20.50010 / 3.10.51000 / 3.10.50000 Note: For the list of the supported Operating Systems, please refer to the driver’s Release Notes. |
MFT | 4.23.0 / 4.22.1 / 4.22.0 Note: For the list of the supported Operating Systems, please refer to the driver’s Release Notes. |
mstflint | 4.23.0 / 4.22.1 / 4.22.0 Note: For the list of the supported Operating Systems, please refer to the driver’s Release Notes. |
FlexBoot | 3.6.901 Note: Please be aware that not all firmware binaries contain FlexBoot or UEFI, support may vary between cards. |
UEFI | 14.29.14 Note: Please be aware that not all firmware binaries contain FlexBoot or UEFI, support may vary between cards. |
MLNX-OS | 3.10.5002 onwards |
NVIDIA Quantum-2 Firmware | 31.2010.5108 onwards |
NVIDIA does not support InfiniBand cables or modules not qualified or approved by NVIDIA.
Switch and HCAs InfiniBand Cable Connectivity Matrix
NVIDIA Quantum™ based switches and NVIDIA® ConnectX® HCAs support NDR (PAM4, 100Gb/s per lane), HDR (PAM4, 50Gb/s per lane) and EDR (NRZ, 25Gb/s per lane) technologies. As the ConnectX adapter cards are identified by their maximum supported throughput (e.g., ConnectX-6 VPI 100Gb/s card can support either 2-lanes of 50Gb/s or 4-lanes of 25Gb/s), the exact connectivity will be determined by the cable that is being used or by manual configuration.
As a reference:
Speed Mode | Speed Supported | Number of Lanes Used |
NDR | 400Gb/s InfiniBand | 4 lanes of 100Gb/s |
NDR200 | 200Gb/s InfiniBand | 2 lanes of 100Gb/s |
The following tables present the connectivity matrix, between NVIDIA Quantum based switches, ConnectX HCA, and the cables.
Switch-to-HCA Connectivity Matrix
Switch | Adapter | Cable | ||||||
NDR Transceiver | NDR ACC/DAC | HDR AOC | HDR DAC | HDR100 DAC/AOC | EDR DAC/AOC | |||
NVIDIA Quantum-2 | NDR Switch | ConnectX-7 400Gb/s | 2 × NDR | 2 × NDR | N/A | N/A | N/A | N/A |
NVIDIA Quantum-2 | ConnectX-7 200Gb/s | 4 × NDR200 | 4 × NDR200 | N/A | N/A | N/A | N/A | |
VPI Protocol Support
ConnectX-7 supports having one port as InfiniBand and the second port as either InfiniBand or Ethernet.
To set the right configuration, run:
mlxconfig -d <mst device> s LINK_TYPE_P1=1/2 LINK_TYPE_P2=1/2
where:
LINK_TYPE_P2 - sets the configuring protocol for port 2
(1/2) - values used for the different protocols:
1 – for InfiniBand
2 - for Ethernet
Validated and Supported NDR Cables
Speed | OPN | Description |
NDR | MCP7Y00-N01A-FLT | NVIDIA passive copper splitter cable, IB twin port NDR 800Gb/s to 2x400Gb/s, OSFP to 2xOSFP,1.5m, flat top |
NDR | MMS4X00-NL-FLT | NVIDIA twin port transceiver, 800Gbps,2xNDR, OSFP, 2xMPO12 APC, 1310nm SMF, up to 30m, flat top |
NDR | MMS4X00-NS-FLT | NVIDIA twin port transceiver, 800Gbps,2xNDR, OSFP, 2xMPO12 APC, 1310nm SMF, up to 100m, flat top |
NDR | MMA4Z00-NS-FLT | NVIDIA twin port transceiver, 800Gbps,2xNDR, OSFP, 2xMPO12 APC, 850nm MMF, up to 50m, flat top |
NDR | MMA1Z00-NS400 | NVIDIA single port transceiver, 400Gbps,NDR, QSFP112, MPO12 APC, 850nm MMF, up to 50m, flat top |
NDR | MCA7J65-N001 | NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 2x400Gb/s, OSFP to 2xQSFP112, 1m |
NDR | MCA7J65-N002 | NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 2x400Gb/s, OSFP to 2xQSFP112, 2m |
NDR | MCA7J65-N003 | NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 2x400Gb/s, OSFP to 2xQSFP112, 3m |
NDR | MCA7J65-N004 | NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 2x400Gb/s, OSFP to 2xQSFP112, 4m |
NDR | MCA7J65-N005 | NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 2x400Gb/s, OSFP to 2xQSFP112, 5m |
NDR | MCA7J75-N001 | NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 4x200Gb/s, OSFP to 4xQSFP112, 1m |
NDR | MCA7J75-N002 | NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 4x200Gb/s, OSFP to 4xQSFP112, 2m |
NDR | MCA7J75-N003 | NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 4x200Gb/s, OSFP to 4xQSFP112, 3m |
NDR | MCA7J75-N004 | NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 4x200Gb/s, OSFP to 4xQSFP112, 4m |
NDR | MCA7J75-N005 | NVIDIA Active copper splitter cable, IB twin port NDR 800Gb/s to 4x200Gb/s, OSFP to 4xQSFP112, 5m |
NDR | MCA4J80-N003-FTF | NVIDIA Active copper cable, IB twin port NDR, up to 800Gb/s, OSFP, 3m, flat to finned |
NDR | MCA4J80-N003-FLT | NVIDIA Active copper cable, IB twin port NDR, up to 800Gb/s, OSFP, 3m, flat to flat |
Validated and Supported HDR Cables
Speed | OPN | Description |
HDR | MFS1S00-HXXXE | Mellanox active fiber cable, IB HDR, up to 200Gb/s, QSFP56, LSZH, black pulltab, Xm |
HDR | MFA7U10-H003 | NVIDIA AOC splitter, IB twin port HDR, 400Gb/s to 2x200Gb/s, OSFP to 2xQSFP56, 3m |
HDR | MFA7U10-H005 | NVIDIA AOC splitter, IB twin port HDR, 400Gb/s to 2x200Gb/s, OSFP to 2xQSFP56, 5m |
HDR | MFA7U10-H010 | NVIDIA AOC splitter, IB twin port HDR, 400Gb/s to 2x200Gb/s, OSFP to 2xQSFP56, 10m |
HDR | MFA7U10-H015 | NVIDIA AOC splitter, IB twin port HDR, 400Gb/s to 2x200Gb/s, OSFP to 2xQSFP56, 15m |
HDR | MFA7U10-H020 | NVIDIA AOC splitter, IB twin port HDR, 400Gb/s to 2x200Gb/s, OSFP to 2xQSFP56, 20m |
HDR | MFA7U10-H030 | NVIDIA AOC splitter, IB twin port HDR, 400Gb/s to 2x200Gb/s, OSFP to 2xQSFP56, 30m |
HDR | MFA7U10-H050 | NVIDIA AOC splitter, IB twin port HDR, 400Gb/s to 2x200Gb/s, OSFP to 2xQSFP56, 50m |
Tested NDR Switches
Speed | Switch Silicon | OPN # / Name | Description | Vendor |
NDR | NVIDIA Quantum-2 | MQM9790 | NVIDIA Quantum-2 based NDR InfiniBand EVB Switch, 64 NDR ports, 32 OSFP ports, non-blocking switching capacity of 51.2Tbps, 2 Power Supplies (AC), Standard depth, Unmanaged, P2C airflow, Rail Kit, RoHS6 | NVIDIA |
NDR | NVIDIA Quantum-2 | MQM9700 | NVIDIA Quantum 2 based NDR InfiniBand Switch, 64 NDR ports, 32 OSFP ports, 2 Power Supplies (AC), Standard depth, Managed, P2C airflow, Rail Kit | NVIDIA |
Tested 400GbE Switches
Speed | Switch Silicon | OPN # / Name | Description | Vendor |
400GbE | Spectrum-3 | MSN4410 | 24 QSFP-DD28 and 8 QSFP-DD ports, 400GbE 1U Open Ethernet Switch with Onyx | NVIDIA |
400GbE | Spectrum-3 | MSN4700 | 32 QSFPDD ports, 400GbE 1U Open Ethernet Switch with Onyx | NVIDIA |
400GbE | N/A | Wedge 400 | Wedge 400-48X 400GbE Data Center Switch | |
400GbE | N/A | Cisco Nexus 3432D-S | Cisco Nexus 3432D-S, 32 fixed 400-Gigabit Ethernet QSFP-DD ports with backward compatibility for QSFP56, QSFP28, and QSFP+ | Cisco |
Tested 200GbE Switches
Speed | Switch Silicon | OPN # / Name | Description | Vendor |
200GbE | Spectrum-3 | MSN4600V-XXXX | 64 QSFP56 ports, 200GbE 2U Open Ethernet Switch with Onyx | NVIDIA |