DPU Mode Installation
DPU mode is the default mode for BlueField DPUs, while BlueField SuperNICs are shipped with NIC mode as their default. To switch between the modes, see NVIDIA BlueField Modes of Operation. To check which mode your BlueField is currently running:
Using Redfish on the 1GbE interface, refer to section "BlueField Modes of Operation Configuration".
From the host server, refer to section "Enabling NIC Mode on BlueField-3 from Linux" in NVIDIA BlueField Modes of Operation.
From the host server's UEFI BIOS menu, refer to section "Configuring NIC Mode on BlueField-3 from Host BIOS HII UEFI Menu" in NVIDIA BlueField Modes of Operation.
In the out-of-box state of the BlueField the host is assumed to be trusted. Later in this procedure, after performing BFB Bundle update, a step is provided to disable the host RShim which the user must perform to protect the BlueField from potential security threats from the host.
The following diagram illustrates the sequence of events and actions from first time power-up of the NVIDIA® BlueField® networking platform (DPU or SuperNIC) in the data center environment through provisioning and maintenance.
If a BlueField-2 is in your possession and it is the first time you are upgrading BlueField BMC, follow the instructions in appendix "BMC and eROT Upgrade Process for BlueField-2".
The numbers indicated in the sequence diagram correspond to the steps that follow it.
At the end of this procedure, the BlueField should be configured with an IP address, all required settings, has up-to-date software component versions, and is ready to use.
The BlueField SoC boots to the UEFI BIOS and DHCP DISCOVER is sent
BlueField SoC runs UEFI/PXE which sends a DHCP DISCOVER over the 1GbE OOB interface, including vendor class (
"NVIDIA/BF/PXE"
) for BlueField SoC (to allow customer's server to differentiate between BlueField SoC and BlueField BMC), and MAC for identification and discovery. See appendix "BlueField DHCP Discover" for more information.A customer's DHCP server inspects the MAC address and the vendor class, allocates IP, and continues the standard DHCP.
DHCP server updates RMC of the new BlueField discovered with detailed information (e.g., MAC, IP address, vendor class).
BlueField BMC issues DHCP DISCOVER over the 1GbE OOB interface, including vendor class ("NVIDIA/BF/BMC"
) for BlueField-BMC, and MAC for identification and discovery. Example of BlueField BMC DHCP DISCOVER packet structure (note "NVIDIA/BF/BMC"
in line 13):
root@bf
-bmc:~# 18
:18
:10.563269
IP (tos 0xc0
, ttl 64
, id 0
, offset 0
, flags [none], proto UDP (17
), length 320
)
0.0
.0.0
.bootpc > 255.255
.255.255
.bootps: [udp sum ok] BOOTP/DHCP, Request from b8:3f:d2:ca:4b:26
(oui Unknown), length 292
, xid 0xfc2acdec
, secs 1
, Flags [none] (0x0000
)
Client-Ethernet-Address b8:3f:d2:ab:cd:ef (oui Unknown)
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message (53
), length 1
: Discover
Client-ID (61
), length 7
: ether b8:3f:d2:ab:cd:ef
Parameter-Request (55
), length 9
:
Subnet-Mask (1
), Default-Gateway (3
), Domain-Name-Server (6
), Hostname (12
)
Domain-Name (15
), Static-Route (33
), NTP (42
), Unknown (120
)
Classless-Static-Route (121
)
MSZ (57
), length 2
: 576
Hostname (12
), length 7
: "bf-bmc"
Vendor-Class (60
), length 13
: "NVIDIA/BF/BMC"
END (255
), length 0
18
:18
:10.565261
IP (tos 0x0
, ttl 63
, id 0
, offset 0
, flags [DF], proto UDP (17
), length 353
)
(example) dhcp01.XX.YY > ldev-platform-13
-043
-bmc.bootpc: [no cksum] BOOTP/DHCP, Reply, length 325
, hops 1
, xid 0xfc2acdec
, secs 1
, Flags [none] (0x0000
)
(example) Your-IP ldev-platform-13
-043
-bmc.XX.YY
(example) Server-IP l-pxe02.XX.YY
Gateway-IP 10.237
.0.255
Client-Ethernet-Address b8:3f:d2:ab:cd:ef (oui Unknown)
file "pxelinux.0"
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message (53
), length 1
: Offer
Server-ID (54
), length 4
: (example) dhcp01.XX.YY
Lease-Time (51
), length 4
: 43200
Subnet-Mask (1
), length 4
: 255.255
.0.0
Default-Gateway (3
), length 4
(example) GW.XX.YY
Hostname (12
), length 24
: "ldev-platform-13-043-bmc"
Domain-Name (15
), length 13
: "<local domain name>"
NTP (42
), length 4
: (example) NTP.XX.YY
END (255
), length 0
18
:18
:10.565261
IP (tos 0x0
, ttl 62
, id 0
, offset 0
, flags [DF], proto UDP (17
), length 353
)
dhcp01.XX.YY > ldev-platform-13
-043
-bmc.<local domain name>: [no cksum] BOOTP/DH
DHCP server inspects the MAC address and the vendor class, allocates IP and continues the standard DHCP flow.
DHCP server updates RMC of the new BlueField BMC discovered with detailed information: MAC, IP address, vendor classes, etc.
To communicate with the BlueField BMC, change the default password (0penBmc
) by sending the following Redfish schema to the BlueField BMC:
curl -k -u root:0penBmc -H "Content-Type: application/json" -X PATCH https://<BF-BMC-IP>/redfish/v1/AccountService/Accounts/root -d '{"Password" : "<user-password>"}'
Where <BF-BMC-IP>
is the IP address for the BlueField BMC (e.g., 10.10.1.2), and <user-password>
is the chosen password to log into the BlueField BMC with root privileges.
For information on the BMC's password policy, refer to section "BMC Password Policy".
For example:
[redfish_scripts] $ curl -k -u root:0penBmc -H "Content-Type: application/json" -X PATCH https://<BF-BMC-IP>/redfish/v1/AccountService/Accounts/root -d '{"Password" : "HelloNvidia3D!"}'
Response:
{
"@Message.ExtendedInfo": [
{
"@odata.type": "#Message.v1_1_1.Message",
"Message": "The request completed successfully.",
"MessageArgs": [],
"MessageId": "Base.1.15.0.Success",
"MessageSeverity": "OK",
"Resolution": "None"
}
]
}
Upgrade the BlueField firmware components (i.e., ATF, UEFI, NIC-firmware, DPU BMC, and ERoT) and the BSP using the BFB image by following the instructions in section "Installing BFB".
Make sure to download the latest DOCA image (BFB file) available from the NVIDIA DOCA Downloader.
Verify BlueField BSP, BlueField BMC and BlueField NIC firmware versions are up to date.
Use the Redfish
FirmwareInventory
schema over the 1GbE OOB interface to the BlueField's BMC:[redfish_scripts] $ curl -k -u root:<password> -H "Content-Type: application/octet-stream" -X GET https://<BF-BMC-IP>/redfish/v1/UpdateService/FirmwareInventory { "@odata.id": "/redfish/v1/UpdateService/FirmwareInventory", "@odata.type": "#SoftwareInventoryCollection.SoftwareInventoryCollection", "Members": [ { "@odata.id": "/redfish/v1/UpdateService/FirmwareInventory/9f7ec75a_BMC_Firmware" }, { "@odata.id": "/redfish/v1/UpdateService/FirmwareInventory/Bluefield_FW_ERoT" }, { "@odata.id": "/redfish/v1/UpdateService/FirmwareInventory/DPU_ATF" }, { "@odata.id": "/redfish/v1/UpdateService/FirmwareInventory/DPU_BOARD" }, { "@odata.id": "/redfish/v1/UpdateService/FirmwareInventory/DPU_BSP" }, { "@odata.id": "/redfish/v1/UpdateService/FirmwareInventory/DPU_NIC" }, { "@odata.id": "/redfish/v1/UpdateService/FirmwareInventory/DPU_NODE" }, { "@odata.id": "/redfish/v1/UpdateService/FirmwareInventory/DPU_OFED" }, { "@odata.id": "/redfish/v1/UpdateService/FirmwareInventory/DPU_OS" }, { "@odata.id": "/redfish/v1/UpdateService/FirmwareInventory/DPU_SYS_IMAGE" }, { "@odata.id": "/redfish/v1/UpdateService/FirmwareInventory/DPU_UEFI" } ], "Members@odata.count": 11, "Name": "Software Inventory Collection" }
Response example for
DPU_ATF
:> curl -k -u root:<password> -H "Content-Type: application/octet-stream" -X GET https://<BF-BMC-IP>/redfish/v1/UpdateService/FirmwareInventory/DPU_ATF { "@odata.id": "/redfish/v1/UpdateService/FirmwareInventory/DPU_ATF", "@odata.type": "#SoftwareInventory.v1_4_0.SoftwareInventory", "Description": "Host image", "Id": "DPU_ATF", "Members@odata.count": 1, "Name": " "Software Inventory", "RelatedItem": [ { "@odata.id": "/redfish/v1/Systems/Bluefield/Bios" } ], "SoftwareId": "", "Status": { "Health": "OK", "HealthRollup": "OK", "State": "OK", }, "Updateable": true, "Version": "v2.2(release):4.0.2-33-gd9f4ad5"
InfoThis request may also be used to query some of the other previously mentioned components (e.g.,
9f7ec75a_BMC_Firmware
,Bluefield_FW_ERoT
).If the versions are not as expected, upgrade as needed:
Download the latest DOCA (BFB file) versions from the downloader at the bottom of the DOCA product page.
DOCA (BFB) upgrade options (upgrading UEFI, ATF, Arm OS, NIC firmware components):
Recommended—BFB upgrade from remote management controller using Redfish
UpdateService
schema over 1GbE to BlueField BMC:export token=`curl -k -H
"Content-Type: application/json"
-X POST https://<bmc_ip>/login -d '{"username":"root", "password":"<password>"}' | grep token | awk '{print $2;}' | tr -d '"'`
For more information on deploying BlueField software from the BMC, refer to the "Deploying BlueField Software Using BFB from BMC" page of the NVIDIA BlueField BSP document.
Get the BlueField's BMC MAC address using the following Redfish command over the 1GbE OOB port to the BlueField BMC:
curl -k -u root:<password> -H 'Content-Type: application/json' -X GET https://<BF-BMC-IP>/redfish/v1/Managers/Bluefield_BMC/EthernetInterfaces/eth0 { "@odata.id": "/redfish/v1/Managers/Bluefield_BMC/EthernetInterfaces/eth0", "@odata.type": "#EthernetInterface.v1_6_0.EthernetInterface", "DHCPv4": { "DHCPEnabled": true, "UseDNSServers": true, "UseDomainName": true, "UseNTPServers": true }, "DHCPv6": { "OperatingMode": "Stateful", "UseDNSServers": true, "UseDomainName": true, "UseNTPServers": true }, "Description": "Management Network Interface", "FQDN": "dpu-bmc", "HostName": "BlueField-bmc", "IPv4Addresses": [ { "Address": "10.237.40.179", "AddressOrigin": "DHCP", "Gateway": "0.0.0.0", "SubnetMask": "255.255.0.0" } ], "IPv4StaticAddresses": [], "IPv6AddressPolicyTable": [], "IPv6Addresses": [ { "Address": "fdfd:fdfd:10:237:966d:aeff:fe17:9f5f", "AddressOrigin": "DHCPv6", "AddressState": null, "PrefixLength": 64 }, { "Address": "fe80::966d:aeff:fe17:9f5f", "AddressOrigin": "LinkLocal", "AddressState": null, "PrefixLength": 64 } ], "IPv6DefaultGateway": "fe80::445b:ed80:5f97:8900", "IPv6StaticAddresses": [], "Id": "eth0", "InterfaceEnabled": true, "LinkStatus": "LinkUp", "MACAddress": "94:6d:ae:17:9f:5f", "MTUSize": 1500, "Name": "Manager Ethernet Interface", "NameServers": [ "fdfd:fdfd:7:77:250:56ff:fe8b:e4f9" ], "SpeedMbps": 0, "StaticNameServers": [], "Status": { "Health": "OK", "HealthRollup": "OK", "State": "Enabled" }, "VLANs": { "@odata.id": "/redfish/v1/Managers/Bluefield_BMC/EthernetInterfaces/eth0/VLANs" } }
Get the BlueField's high-speed port's MAC addresses using the following Redfish command over the 1GbE OOB port to the BlueField BMC:
curl -k -u root:<password> -H "Content-Type: application/octet-stream" -X GET https://<bmc_ip>/redfish/v1/Chassis/Card1/NetworkAdapters/NvidiaNetworkAdapter/NetworkDeviceFunctions/eth0f0 { "@odata.id": "/redfish/v1/Chassis/Card1/NetworkAdapters/NvidiaNetworkAdapter/NetworkDeviceFunctions/eth0f0", "@odata.type": "#NetworkDeviceFunction.v1_9_0.NetworkDeviceFunction", "Ethernet": { "MACAddress": "02:b1:b6:12:39:05", "MTUSize": 1500 }, "Id": "eth0f0", "Links": { "OffloadSystem": { "@odata.id": "/redfish/v1/Systems/Bluefield" }, "PhysicalPortAssignment": { "@odata.id": "/redfish/v1/Chassis/Card1/NetworkAdapters/NvidiaNetworkAdapter/Ports/eth0" } }, "Name": "NetworkDeviceFunction", "NetDevFuncCapabilities": [ "Ethernet" ], "NetDevFuncType": "Ethernet" }
Unless it is explicitly desired for the host to be trusted, make sure to disable the host PCIe RShim to protect the BlueField from potential security threats from the host:
Use Redfish BIOS settings schema over the 1GbE OOB to the BlueField BMC:
curl -k -X PATCH -d '{"Attributes":{"Internal CPU Model": "Restricted"}}' -u root:<password> https://<BF-BMC-IP>/redfish/v1/Systems/<SystemID>/Bios/Settings | python3 -m json.tool
The available BlueField host privilege levels are
Restricted
andPrivileged
. The default isPrivileged
, where the host has access to BlueField.Change the privilege level to
Restricted
.
Changing host privilege level requires BlueField DPU reset and host power cycle for the change to take effect.
For more information on BlueField modes of operation, refer to this page.
As part of the default settings of the BlueField, UEFI Secure Boot is enabled and requires no special configuration to use it with the bundled Ubuntu OS shipped with the BlueField device. Disabling UEFI Secure Boot may be necessary when running an unsigned Arm OS image, such as a customer OS.
To disable secure boot using the Redfish SecureBoot
schema over 1GbE to BlueField BMC, follow the command in section "Setting Secure Boot State".
For more information on user management, review this page.