NVIDIA BlueField BSP v4.8.0
NVIDIA BlueField BSP v4.8.0 Download PDF

NVIDIA BlueField Reset and Reboot Procedures

This section describes the necessary operations to load new NIC firmware, following NVIDIA® BlueField® NIC firmware update. This procedure deprecates the need for full server power cycle.

The following steps are executed in the BlueField OS:

  1. Issue a query command to ascertain whether BlueField system reboot is supported by your environment:

    Copy
    Copied!
                

    mlxfwreset -d 03:00.0

    If the output includes the following lines, proceed to step 2:

    Copy
    Copied!
                

    3: Driver restart and PCI reset         -Supported (default) ... 1: Driver is the owner                  -Supported (default)

    Note

    If it says Not Supported instead, then proceed to the instructions under section "BlueField System-level Reset".

  2. Issue a BlueField system reboot:

    Copy
    Copied!
                

    mlxfwreset -d 03:00.0 -y -l 3 --sync 1 r

This section describes the way to perform system-level reset (SLR) which is necessary for firmware configuration changes to take effect.

System-level Reset for BlueField in DPU Mode

The following is the high-level flow of the procedure:

  1. Graceful shutdown of BlueField Arm cores.

  2. Query BlueField state to affirm shutdown reached.

    Info

    In systems with multiple BlueField networking platforms, repeat steps 1 and 2 for all devices before proceeding.

  3. Warm reboot the server.

Step by step process:

Info

Some of the following steps can be performed using different methods, depending on resource availability and support in the user's environment.

  1. Graceful shutdown of BlueField Arm cores.

    Info

    This operation is expected to finish within 15 seconds.

    Possible methods:

    • From the BlueField OS:

      Copy
      Copied!
                  

      shutdown -h now

      Or:

      Copy
      Copied!
                  

      mlxfwreset -d /dev/mst/mt*pciconf0 -l 1 -t 4 --sync 0 r

    • From the host OS:

      Info

      Not relevant when the BlueField is operating in Zero-Trust Mode.

      Copy
      Copied!
                  

      mlxfwreset -d <mst-device> -l 1 -t 4 r

    • Using the BlueField BMC:

      Copy
      Copied!
                  

      ipmitool -C 17 -I lanplus -H <bmc_ip> -U root -P <password> power soft

      Or using Redfish (BlueField-3 and above):

      Copy
      Copied!
                  

      curl -k -u root:<password> -H "Content-Type: application/json" -X POST https://<bmc_ip>/redfish/v1/Systems/Bluefield/Actions/ComputerSystem.Reset -d '{"ResetType": "GracefulShutdown"}'

  2. Query BlueField state.

    Possible methods:

    • From the host OS:

      Info

      Not relevant when the BlueField is operating in Zero-Trust Mode.

      Copy
      Copied!
                  

      echo DISPLAY_LEVEL 2 > /dev/rshim0/misc cat /dev/rshim0/misc

      Expected output:

      Copy
      Copied!
                  

      INFO[BL31]: System Off

    • Utilizing the BlueField BMC:

      Copy
      Copied!
                  

      ipmitool -C 17 -I lanplus -H <bmc_ip> -U root -P <password> raw 0x32 0xA3

      Expected output: 06.

  3. Warm reboot the server f rom the host OS:

    Copy
    Copied!
                

    mlxfwreset -d <mst-device> -l 4 r

    Note

    If multiple BlueField devices are present in the host, this command must run only once. In this case, the MST device can be of any of the BlueFields for which the reset is necessary and participated in step 1.

    Or:

    Copy
    Copied!
                

    reboot

    Note

    For external hosts which do not toggle PERST# in their standard reboot command, use the mlxfwreset option.

System-level Reset for BlueField in NIC Mode

Perform warm reboot of the host OS:

Copy
Copied!
            

mlxfwreset -d <mst-device> -l 4 r

Or:

Copy
Copied!
            

reboot

Note

For external hosts which do not toggle PERST# in their standard reboot command, use the mlxfwreset option.


System-level Reset for Host with Separate Power Control

This procedure is a special use case relevant only to host platforms with separate power control for the PCIe slot and CPUs, in which the BlueField (running in DPU mode) is provided power while host OS/CPUs may be in shutdown or similar standby state (this allows the BlueField device to be operational while the host CPU is in shutdown/standby state).

The following is the high-level flow of the procedure:

  1. Graceful shutdown of host OS or similar CPU standby.

  2. Graceful shutdown of BlueField Arm cores.

  3. Query BlueField state to affirm shutdown reached.

  4. Full BlueField Reset

  5. Query BlueField state to affirm operational state reached

    Info

    In systems with multiple BlueField networking platforms, repeat steps 1 through 5 for all devices before proceeding.

  6. Power on the server.

Step by step process:

Info

Some of the following steps can be performed using different methods, depending on resource availability and support in the user's environment.

  1. Graceful shutdown of host OS by any means preferable.

  2. Graceful shutdown of BlueField Arm cores.

    Info

    This step normally takes up to 15 seconds to complete.

    • From the BlueField OS:

      Copy
      Copied!
                  

      shutdown -h now

    • Utilizing the BlueField BMC:

      • Using IPMI:

        Copy
        Copied!
                    

        ipmitool -C 17 -I lanplus -H <bmc_ip> -U root -P <password> power soft

      • Using Redfish (for BlueField-3 and above):

        Copy
        Copied!
                    

        curl -k -u root:<password> -H "Content-Type: application/json" -X POST https://<bmc_ip>/redfish/v1/Systems/Bluefield/Actions/ComputerSystem.Reset -d '{"ResetType": "GracefulShutdown"}'

  3. Query the BlueField's state utilizing the BlueField BMC:

    Copy
    Copied!
                

    ipmitool -C 17 -I lanplus -H <bmc_ip> -U root -P <password> raw 0x32 0xA3

    Expected output: 06.

  4. Perform BlueField hard reset utilizing the BlueField BMC:

    Info

    This step takes up to 2 minutes to complete .

    • Using IPMI:

      Copy
      Copied!
                  

      ipmitool -C 17 -I lanplus -H <bmc_ip> -U root -P <password> power cycle

    • Using Redfish (for BlueField-3 and above):

      Copy
      Copied!
                  

      curl -k -u root:<password> -H "Content-Type: application/json" -X POST https://<bmc_ip>/redfish/v1/Systems/Bluefield/Actions/ComputerSystem.Reset -d '{"ResetType" : "PowerCycle"}'

  5. Query BlueField operational state u tilizing the BlueField BMC :

    Info

    At this point, the BlueField is expected to b e operational .

    Copy
    Copied!
                

    ipmitool -C 17 -I lanplus -H <bmc_ip> -U root -P <password> raw 0x32 0xA3

    Expected output: 05.

  6. Power on/boot up the host OS.

© Copyright 2024, NVIDIA. Last updated on Aug 20, 2024.