Base OS - DGX OS 5

Appendix B: Downgrade Firmware for Mellanox ConnectX-4 Cards

DGX OS 5.0 provides the mlnx-fw-updater package version 5.1-2.4.6.0 which automatically installs firmware version 12.28.2040 on ConnectX-4 devices.

Since 12.28.2006 is the recommended firmware version, on December 15 the updater package was updated to install version 12.28.2006. However, if the firmware has already been updated to 12.28.2040, the updater will not install the downlevel firmware version since a newer version is already installed.

In this case, you will need to force the downgrade as explained in this section.

You can use the mlxfwmanager tool to verify whether ConnectX-4 devices are installed on your DGX system.

Run the following command:

Copy
Copied!
            

$ sudo mlxfwmanager Querying Mellanox devices firmware ... Device #1: Device Type: ConnectX4 Part Number: MCX455A-ECA_Ax Description: ConnectX-4 VPI adapter card; EDR IB (100Gb/s) and 100GbE; single-port QSFP28; PCIe3.0 x16; ROHS R6 PSID: MT_2180110032 PCI Device Name: /dev/mst/mt4115_pciconf1 Base GUID: 248a070300945e60 Versions: Current Available FW 12.28.2040 N/A PXE 3.6.0102 N/A UEFI 14.21.0017 N/A

If the output indicates that ConnectX-4 devices are installed, you need to downgrade the firmware.

To downgrade the firmware:

  1. Determine the correct firmware package name.

  2. Switch to the /opt/Mellanox/mlnx-fw-updater/firmware directory, where the updater installs the firmware files, and list the contents.

    Copy
    Copied!
                

    $ ls /opt/mellanox/mlnx-fw-updater/firmware

  3. Identify the correct package from the output.

    Copy
    Copied!
                

    mlxfwmanager_sriov_dis_x86_64_4115 mlxfwmanager_sriov_dis_x86_64_4119 mlxfwmanager_sriov_dis_x86_64_4123 mlxfwmanager_sriov_dis_x86_64_4127 mlxfwmanager_sriov_dis_x86_64_41686 mlxfwmanager_sriov_dis_x86_64_4117 mlxfwmanager_sriov_dis_x86_64_4121 mlxfwmanager_sriov_dis_x86_64_4125 mlxfwmanager_sriov_dis_x86_64_41682

  4. Execute the firmware package by using the -f flag.

    Copy
    Copied!
                

    $ cd /opt/mellanox/mlnx-fw-updater/firmware

    Copy
    Copied!
                

    $ sudo ./mlxfwmanager_sriov_dis_x86_64_4115 -f The software queries the current firmware and then updates (downgrades) the firmware. Querying Mellanox devices firmware … Found 2 device(s) requiring firmware update... Device #1: Updating FW … Initializing image partition - OK Writing Boot image component - OK Done Device #2: Updating FW … Initializing image partition - OK Writing Boot image component - OK Done

  5. Reboot the system to allow the updates to take effect.

    Copy
    Copied!
                

    $ sudo reboot

© Copyright 2020-2023, NVIDIA. Last updated on Mar 24, 2023.