Downgrading ConnectX Firmware#
DGX OS 5.0 provides the mlnx-fw-updater package version 5.1-2.4.6.0 which automatically installs firmware version 12.28.2040 on ConnectX-4 devices.
Since 12.28.2006 is the recommended firmware version, on December 15 the updater package was updated to install version 12.28.2006. However, if the firmware has already been updated to 12.28.2040, the updater will not install the downlevel firmware version since a newer version is already installed.
In this case, you will need to force the downgrade as explained in this section.
Checking the Device Type#
You can use the mlxfwmanager tool to verify whether ConnectX-4 devices are installed on your DGX system.
Run the following command:
sudo mlxfwmanager
Querying Mellanox devices firmware ... Device #1:
Device Type: ConnectX4
Part Number: MCX455A-ECA_Ax
Description: ConnectX-4 VPI adapter card; EDR IB (100Gb/s) and 100GbE;
single-port QSFP28; PCIe3.0 x16; ROHS R6
PSID: MT_2180110032
PCI Device Name: /dev/mst/mt4115_pciconf1 Base GUID: 248a070300945e60
Versions: Current Available FW 12.28.2040 N/A
PXE 3.6.0102 N/A
UEFI 14.21.0017 N/A
Downgrading the Firmware#
If the output indicates that ConnectX-4 devices are installed, you need to downgrade the firmware.
To downgrade the firmware:
Determine the correct firmware package name.
Switch to the
/opt/Mellanox/mlnx-fw-updater/firmware
directory, where the updater installs the firmware files, and list the contents.ls /opt/mellanox/mlnx-fw-updater/firmware
Identify the correct package from the output.
mlxfwmanager_sriov_dis_x86_64_4115 mlxfwmanager_sriov_dis_x86_64_4119 mlxfwmanager_sriov_dis_x86_64_4123 mlxfwmanager_sriov_dis_x86_64_4127 mlxfwmanager_sriov_dis_x86_64_41686 mlxfwmanager_sriov_dis_x86_64_4117 mlxfwmanager_sriov_dis_x86_64_4121 mlxfwmanager_sriov_dis_x86_64_4125 mlxfwmanager_sriov_dis_x86_64_41682
Execute the firmware package by using the
-f
flag.cd /opt/mellanox/mlnx-fw-updater/firmware
sudo ./mlxfwmanager_sriov_dis_x86_64_4115 -f The software queries the current firmware and then updates (downgrades) the firmware. Querying Mellanox devices firmware … Found 2 device(s) requiring firmware update... Device #1: Updating FW … Initializing image partition - OK Writing Boot image component - OK Done Device #2: Updating FW … Initializing image partition - OK Writing Boot image component - OK Done
Reboot the system to allow the updates to take effect.
sudo reboot