NVIDIA BlueField Reset and Reboot Procedures
This section describes the necessary operations to load new NIC firmware, following NVIDIA® BlueField® NIC firmware update. This procedure deprecates the need for full server power cycle.
The following steps are executed in the BlueField OS:
Issue a query command to ascertain whether BlueField system reboot is supported by your environment:
mlxfwreset -d
03:00.0qIf the output includes the following lines, proceed to step 2:
3: Driver restart and PCI reset -Supported (default) ...1: Driver is the owner -Supported (default)NoteIf it says Not Supported instead, then proceed to the instructions under section "BlueField System-level Reset".
Issue a BlueField system reboot:
mlxfwreset -d
03:00.0-y -l3--sync1r
This section describes the necessary system-level reset following firmware configuration changes.
The two methods for performing BlueField system-level reset are described in the following subsection. Each method is designed to support different host platforms, in which host OS/CPUs and PCIe slots may have uniform or separate power control.
In each approach, the procedure can be performed through various methods, according to resource availability and support in the user's environment.
System-level Reset for BlueField in DPU Mode with Minimal Host OS Downtime
The following is the high-level flow of the procedure:
Graceful shutdown of BlueField Arm cores.
Query BlueField state to affirm shutdown reached.
InfoIn systems with multiple BlueField networking platforms, repeat steps 1 and 2 for all devices before proceeding.
Warm reboot the server.
Step by step process:
Graceful shutdown of BlueField Arm cores.
InfoThis operation is expected to finish within 15 seconds.
Options:
From the BlueField OS:
shutdown -h now
Or:
mlxfwreset -d /dev/mst/mt*pciconf0 -l
1-t4--sync0rFrom the host OS:
InfoNot relevant when the BlueField is operating in Zero-Trust Mode.
mlxfwreset -d <mst-device> -l
1-t4rUsing the BlueField BMC:
ipmitool -C
17-I lanplus -H <bmc_ip> -U root -P <password> power softOr using Redfish (BlueField-3 and above):
curl -k -u root:<password> -H
"Content-Type: application/json"-X POST https://<bmc_ip>/redfish/v1/Systems/Bluefield/Actions/ComputerSystem.Reset -d '{"ResetType": "GracefulShutdown"}'
Query BlueField state. Options:
From the host OS:
InfoNot relevant when the BlueField is operating in Zero-Trust Mode.
echo DISPLAY_LEVEL
2> /dev/rshim0/misc cat /dev/rshim0/miscExpected output:
INFO[BL31]: System Off
Utilizing the BlueField BMC:
ipmitool -C
17-I lanplus -H <bmc_ip> -U root -P <password> raw0x320xA3Expected output: 06.
Warm reboot the server:
From the host OS:
mlxfwreset -d <mst-device> -l
4rNoteIf multiple DPUs are present in the host, this command must run only once. In this case, the MST device can be of any of the DPUs for which the reset is necessary and participated in step 1.
Or:
reboot
NoteFor external hosts which do not toggle PERST# in their standard reboot command, use the mlxfwreset option.
System-level Reset for BlueField in DPU Mode where Host is Down Throughout the Process
This procedure is only relevant to server platforms that have separate power control for PCIe slot and CPUs in which the BlueField is provided power while host OS/CPUs may be in shutdown or similar standby state.
The following is the high-level flow of the procedure:
Graceful shutdown of host OS or similar CPU standby.
Graceful shutdown of BlueField Arm cores.
Query BlueField state to affirm shutdown reached.
Full BlueField Reset
Query BlueField state to affirm operational state reached
InfoIn systems with multiple BlueField networking platforms, repeat steps 1 through 5 for all devices before proceeding.
Power on the server.
Step by step process:
Graceful shutdown of host OS by any means preferable.
Graceful shutdown of BlueField Arm cores.
InfoThis step normally takes up to 15 seconds to complete.
From the BlueField OS:
shutdown -h now
Utilizing the BlueField BMC:
Using IPMI:
ipmitool -C
17-I lanplus -H <bmc_ip> -U root -P <password> power softUsing Redfish (for BlueField-3 and above):
curl -k -u root:<password> -H
"Content-Type: application/json"-X POST https://<bmc_ip>/redfish/v1/Systems/Bluefield/Actions/ComputerSystem.Reset -d '{"ResetType": "GracefulShutdown"}'
Query the BlueField's state utilizing the BlueField BMC:
ipmitool -C
17-I lanplus -H <bmc_ip> -U root -P <password> raw0x320xA3Expected output: 06.
Perform BlueField hard reset utilizing the BlueField BMC:
InfoThis step takes up to 2 minutes to complete .
Using IPMI:
ipmitool -C
17-I lanplus -H <bmc_ip> -U root -P <password> power cycleUsing Redfish (for BlueField-3 and above):
curl -k -u root:<password> -H
"Content-Type: application/json"-X POST https://<bmc_ip>/redfish/v1/Systems/Bluefield/Actions/ComputerSystem.Reset -d '{"ResetType" : "PowerCycle"}'
Query BlueField operational state u tilizing the BlueField BMC :
InfoAt this point, the BlueField is expected to b e operational .
ipmitool -C
17-I lanplus -H <bmc_ip> -U root -P <password> raw0x320xA3Expected output: 05.
Power on/boot up the host OS.
System-level Reset for BlueField in NIC Mode
Perform warm reboot of the host OS:
mlxfwreset -d <mst-device> -l 4 r
Or:
reboot
For external hosts which do not toggle PERST# in their standard reboot command, use the mlxfwreset option.