NVIDIA Docs Hub NVIDIA Networking BlueField DPUs / SuperNICs & DOCA DOCA Documentation v2.10.0 BlueField Modes of Operation

BlueField Modes of Operation

This document describes the modes of operation available for NVIDIA® BlueField® networking platforms (DPUs or SuperNICs).

Introduction

BlueField devices feature the following modes of operation:

Mode of Operation	External Host Trust Level	Description	Default on SKUs	Can be Configured to All Other Modes?
NIC Mode	Host-trusted	The Arm cores of BlueField are inactive, and the device functions as an NVIDIA® ConnectX® network adapter.	SuperNIC SKUs default mode	Yes¹
DPU Mode	Host-trusted	The Arm cores of BlueField are active, and the embedded Arm system runs services that manage the NIC resources and data path.	DPU SKUs default mode	Yes
DPU Mode	Zero Trust (restricted)	The Arm cores of BlueField are active, and the embedded Arm system runs services to manage the NIC resources and data path while enforcing restrictions on the external host (host isolation).	-	Yes

BFx H20 converged accelerator is limited to NIC Mode only.

NIC Mode

In NIC Mode, BlueField operates as a ConnectX network adapter for the external host. For BlueField-3, the Arm cores are inactive, while for BlueField-2, the Arm cores are active but non-functional.

Operating in NIC Mode on BlueField-3 reduces power consumption, improves network performance, and minimizes the host memory footprint.

Note

BlueField-3 SuperNIC SKUs are shipped in NIC Mode by default.

Note

Multi-host is not supported when BlueField is operating in NIC Mode.

DPU Mode

In this operation mode, Arm cores active, known also as embedded CPU function ownership (ECPF) mode, is the default mode for the BlueField DPU family of SKUs.

In DPU Mode, the NIC resources and functionality are owned and controlled by the embedded Arm subsystem. All network communication to the host flows through a virtual switch control plane hosted on the Arm cores that manages all networking traffic coming and going from the host.

While working in this mode, the BlueField is the trusted function managed by the data center and host administrator for provisioning, management and orchestration (eg. load network drivers, reset an interface, bring an interface up and down, update the firmware, change the mode of operation on BlueField, etc).

Note

BlueField-2 DPU and BlueField-3 DPU SKUs are shipped in DPU Mode by default.

Note

Socket Direct is not supported when BlueField is operating in DPU Mode.

DPU Mode Architecture

In DPU Mode, the BlueField DPU provides the host system with access to network functions, while the host's capabilities are restricted and managed by the Arm processor within the BlueField.

embedded-mode-version-1-modificationdate-1733837115003-api-v2.png

Traffic Management and Paths

The Embedded Control and Processing Framework (ECPF) controls the NIC's embedded switch (eswitch). All network traffic between the host interface and the network initially passes through the BlueField’s Arm processor via Representors. This path, where traffic is processed by the Arm processor, is referred to as the "slow path".

To improve performance, the Arm processor can define rules in the eswitch through the ECPF, allowing packets to bypass the Arm processor and be processed directly by the eswitch. This is known as the "fast path", which reduces latency and increases throughput by offloading traffic processing from the Arm to the eswitch.

A virtual switch running on the Arm processor may integrate both slow path and fast path functionalities by processing and classifying only the first packet of a new flow. For subsequent packets in the flow, the virtual switch defines eswitch rules, enabling fast path processing for the remainder of the traffic.

Initialization Process

At startup, network access to the host is initially blocked. This restriction remains until the virtual switch running on the Arm processor loads the default out-of-box rules to manage the ECPF on the BlueField. Once these rules are loaded, network traffic to the host is automatically enabled.

Note

The driver on the host system can only be loaded after the driver on the BlueField has been loaded and has completed NIC configuration. Additionally, all Interface Configuration Memory (ICM) is allocated by the ECPF and resides in the BlueField's memory.

InfiniBand in DPU Mode

In DPU Mode, when operating with an InfiniBand network, OpenSM must be executed from the BlueField Arm side rather than the host side. Similarly, InfiniBand management tools such as sminfo, ibdev2netdev, and ibnetdiscover can only be used from the BlueField Arm side and are not accessible from the host side.

DPU Mode in Zero Trust

Zero Trust, also known as Restricted Mode, is a specialized variation of DPU Mode that enhances security by preventing the host system administrator from accessing BlueField from the host side. Once Zero Trust mode is enabled, the BlueField must be fully controlled by the data center administrator via the Arm cores or the BMC connection, rather than through the host.

This mode enforces security and isolation by restricting the host from performing operations that could compromise BlueField. The following operations can be restricted individually in Zero Trust mode:

Port ownership – The host cannot assign itself as the port owner
Hardware counters – The host is denied access to hardware counters
Tracer functionality – The tracer functionality is blocked
RShim interface – The RShim interface is disabled
Firmware flash – firmware flashing from the host is restricted

Zero Trust mode ensures a robust security boundary between the host and BlueField, making it an ideal configuration for environments requiring strict control and isolation.

Moving Between Operation Modes

Transitioning between operation modes can be achieved through various configuration interfaces as presented in the following table.

Note

Some configuration interfaces may be locked out when operating in Zero Trust (Restricted) Mode.

Interface	From	To	Configuration option available by default, for this interface?	Configuration option can be locked-out²for this interface?
External Host command line	DPU Mode	NIC Mode	Yes	Yes
	NIC Mode	DPU Mode	Yes	Yes
	DPU Mode	DPU Mode with Zero Trust	No	Always blocked
	DPU Mode with Zero Trust	DPU Mode	No	Always blocked
	DPU Mode with Zero Trust	NIC Mode	Not supported	N/A
Host UEFI Menu	DPU Mode	NIC Mode	Yes	Yes
	NIC Mode	DPU Mode	Yes	Yes
	DPU Mode	DPU Mode with Zero Trust	No	Always blocked
	DPU Mode with Zero Trust	DPU Mode	No	Always blocked
	DPU Mode with Zero Trust	NIC Mode	Not supported	N/A
Arm OS command line	DPU Mode	NIC Mode	Yes	No
	NIC Mode	DPU Mode	N/A	N/A
	DPU Mode	DPU Mode with Zero Trust	Yes	No
	DPU Mode with Zero Trust	DPU Mode	Yes	No
	DPU Mode with Zero Trust	NIC Mode	Not supported	N/A
Arm UEFI Menu	DPU Mode	NIC Mode	Yes	No
	NIC Mode	DPU Mode	Yes	No
	DPU Mode	DPU Mode with Zero Trust	No ³	No
	DPU Mode with Zero Trust	DPU Mode	No ³	No
	DPU Mode with Zero Trust	NIC Mode	Not supported	N/A
DPU-BMC Redfish	DPU Mode	NIC Mode	Yes	No
	NIC Mode	DPU Mode	Yes	No
	DPU Mode	DPU Mode with Zero Trust	No ³	No
	DPU Mode with Zero Trust	DPU Mode	No ³	No
	DPU Mode with Zero Trust	NIC Mode	Not supported	N/A
Platform-BMC NC-SI OEM commands	DPU Mode	NIC Mode	Yes	No
	NIC Mode	DPU Mode	Yes	No
	DPU Mode	DPU Mode with Zero Trust	No ³	No
	DPU Mode with Zero Trust	DPU Mode	No ³	No
	DPU Mode with Zero Trust	NIC Mode	Not supported	N/A

Blocked or restricted
Roadmap

Identifying Which Mode BlueField is Currently Operating In

Interface

Command

Response

Using external host command line

Copy
Copied!

            
            host> sudo mlxconfig -d /dev/mst/<device> q INTERNAL_CPU_OFFLOAD_ENGINE

Output format:

Copy
Copied!

            
            RO     INTERNAL_CPU_OFFLOAD_ENGINE     ENABLED(0)

ENABLED(0) means BlueField is running in DPU Mode
DISABLED(1) means BlueField is running in NIC Mode
RO is marked for read only, and indicates DPU Mode with Zero Trust

Using external host UEFI (HII) menu

Navigate to specific device and select BlueField Internal Cpu Configuration.

bluefield-internal-cpu-configuration-version-1-modificationdate-1733837114770-api-v2.png

Internal Cpu Offload Engine indicates the mode BlueField is currently operating in:

Disabled – BlueField is operating in NIC Mode
Enabled – BlueField is operating in DPU Mode

internal-cpu-offload-engine-version-1-modificationdate-1733837114550-api-v2.png

Using Arm UEFI (GUI) menu

Access the Arm UEFI menu by pressing the Esc button twi ce on the console and n avigate to Device Manager → System Configuration → BlueField Modes .

The NIC Mode menu indicates the mode BlueField is currently operating in:

NicMode – BlueField is operating in NIC Mode
DpuMode – BlueField is operating in DPU Mode

image-2024-12-23_18-52-9-version-1-modificationdate-1734972728993-api-v2.png

Using DPU BMC Redfish

Copy
Copied!

            
            curl -k -u root:'<PASSWORD>' -X GET https://<bmc_ip>/redfish/v1/Systems/Bluefield/Oem/Nvidia

Read Mode field:

Copy
Copied!

            
            {
...
"Mode": <"DpuMode"> or <"NicMode">
...
}

"Mode": NicMode – BlueField is operating in NIC Mode
"Mode":DpuMode – BlueField is operating in DPU Mode

Using platform BMC NC-SI OEM commands

Get Command = 0x13, Parameter = 0x33

Offset	31:24	23:16	15:8	7:0
0:15	NC-SI OEM Header (OEM Command)
16:19	Mellanox Manufacture ID (IANA) = 0x8119
20:23	Command rev=0x00	MLNX Cmd ID= 0x13	Parameter=0x33	Reserved
24:27	Checksum 31:0

Response format:

Offset	31:24	23:16	15:8	7:0
0:15	NC-SI OEM Header (OEM Command)
16:19	Response Code		Reason Code
20:23	Mellanox Manufacture ID (IANA) = 0x8119
24:27	Command rev=0x00	MLNX Cmd ID= 0x12	Parameter=0x33	Reserved
28:31					Offload engine
32:35	Checksum 31:0

Response field:

Field

Size

Offset in NC-SI Command

Description

Offload engine

1 bit

31.0

0x0: Enabled - DPU Mode

0x1: Disabled - NIC Mode

Changing Mode

For the configuration to take effect, Arm and NIC components must undergo a reset. Power cycle is recommended.

Using External Host Command Line

From Mode	To Mode	Command
DPU Mode	NIC Mode	For BlueField-3: Copy Copied! `host> sudo mlxconfig -d /dev/mst/<device> s INTERNAL_CPU_OFFLOAD_ENGINE=1` For BlueField-2: Copy Copied! `host> sudo mlxconfig -d /dev/mst/<device> s INTERNAL_CPU_PAGE_SUPPLIER=1 INTERNAL_CPU_ESWITCH_MANAGER=1 INTERNAL_CPU_IB_VPORT0=1 INTERNAL_CPU_OFFLOAD_ENGINE=1`
NIC Mode	DPU Mode	For BlueField-3: Copy Copied! `host> sudo mlxconfig -d /dev/mst/<device> s INTERNAL_CPU_OFFLOAD_ENGINE=0` For BlueField-2: Copy Copied! `host> sudo mlxconfig -d /dev/mst/<device> s INTERNAL_CPU_PAGE_SUPPLIER=0 INTERNAL_CPU_ESWITCH_MANAGER=0 INTERNAL_CPU_IB_VPORT0=0 INTERNAL_CPU_OFFLOAD_ENGINE=0`
DPU Mode	DPU Mode with Zero Trust	N/A
DPU Mode with Zero Trust	DPU Mode	N/A
DPU Mode with Zero Trust	NIC Mode	Not supported. Move from DPU Mode with Zero Trust to DPU Mode first, and then from DPU Mode to NIC Mode. Warning Perform a system-level reset when moving from Zero Trust to DPU Mode before configuring NIC Mode. Ensure to disable Zero Trust configuration before transitioning to NIC Mode. Operating in NIC Mode with Zero Trust (Restricted) configuration is not supported and may lead to undefined behavior.

Using External Host UEFI Menu

From Mode	To Mode	Command
DPU Mode	NIC Mode	Select `BlueField Internal Cpu Configuration`. To enable NIC Mode, set `Internal Cpu Offload Engine` to `Disabled`.
NIC Mode	DPU Mode	Select `BlueField Internal Cpu Configuration`. To enable DPU Mode, set `Internal Cpu Offload Engine` to `Enabled`.
DPU Mode	DPU Mode with Zero Trust	N/A
DPU Mode with Zero Trust	DPU Mode	N/A
DPU Mode with Zero Trust	NIC Mode	Not supported. Move from DPU Mode with Zero Trust to DPU Mode first, and then from DPU Mode to NIC Mode. Warning Perform a system-level reset when moving from Zero Trust to DPU Mode before configuring NIC Mode. Ensure to disable Zero Trust configuration before transitioning to NIC Mode. Operating in NIC Mode with Zero Trust (Restricted) configuration is not supported and may lead to undefined behavior.

Using Arm OS command line

From Mode	To Mode	Command
DPU Mode	NIC Mode	For BlueField-3: Copy Copied! `bf> sudo mlxconfig -d /dev/mst/<device> s INTERNAL_CPU_OFFLOAD_ENGINE=1` For BlueField-2: Copy Copied! `bf> sudo mlxconfig -d /dev/mst/<device> s INTERNAL_CPU_PAGE_SUPPLIER=1 INTERNAL_CPU_ESWITCH_MANAGER=1 INTERNAL_CPU_IB_VPORT0=1 INTERNAL_CPU_OFFLOAD_ENGINE=1`
NIC Mode	DPU Mode	Not Applicable (Arm OS is not available)
DPU Mode	DPU Mode with Zero Trust	Copy Copied! `bf> $ sudo mlxprivhost -d /dev/mst/<device> r --disable_rshim --disable_tracer --disable_counter_rd --disable_port_owner`
DPU Mode with Zero Trust	DPU Mode	Copy Copied! `bf> $ sudo mlxprivhost -d /dev/mst/<device> p`
DPU Mode with Zero Trust	NIC Mode	Not supported. Move from DPU Mode with Zero Trust to DPU Mode first, and then from DPU Mode to NIC Mode. Warning Perform a system-level reset when moving from Zero Trust to DPU Mode before configuring NIC Mode. Ensure to disable Zero Trust configuration before transitioning to NIC Mode. Operating in NIC Mode with Zero Trust (Restricted) configuration is not supported and may lead to undefined behavior.

Using Arm UEFI Menu

From Mode	To Mode	Command
DPU Mode	NIC Mode	Access the Arm UEFI menu by pressing the Esc button twice on the console. Select `Device Manager` → `System Configuration` → `BlueField Modes`. Set the `NIC Mode` option to `NicMode` to enable NIC Mode. Exit `BlueField Modes` and `System Configuration` and make sure to save the settings. Exit the UEFI setup using the "reset" option. The configuration is not yet applied and BlueField is expected to boot in DPU Mode. Issue power cycle to apply new configuration.
NIC Mode	DPU Mode	Access the Arm UEFI menu by pressing the Esc button twice on the console. Select `Device Manager` → `System Configuration` → `BlueField Modes`. Set the `NIC Mode` option to `DpuMode` to enable DPU Mode. Exit `BlueField Modes` and `System Configuration` and make sure to save the settings. Exit the UEFI setup using the "reset" option. The configuration is not yet applied and BlueField is expected to boot in NIC Mode. Issue power cycle to apply new configuration.
DPU Mode	DPU Mode with Zero Trust	Roadmap
DPU Mode with Zero Trust	DPU Mode	Roadmap
DPU Mode with Zero Trust	NIC Mode	Not supported. Move from DPU Mode with Zero Trust to DPU Mode first, and then from DPU Mode to NIC Mode. Warning Perform a system-level reset when moving from Zero Trust to DPU Mode before configuring NIC Mode. Ensure to disable Zero Trust configuration before transitioning to NIC Mode. Operating in NIC Mode with Zero Trust (Restricted) configuration is not supported and may lead to undefined behavior.

Using DPU-BMC Redfish

From Mode	To Mode	Command
DPU Mode	NIC Mode	Copy Copied! `curl -k -u root:'password' -H 'content-type: application/json' -d '{"Mode": "NicMode"}' -X POST https://bmc_ip/redfish/v1/Systems/Bluefield/Oem/Nvidia/Actions/Mode.Set` To apply configuration, two consecutive Arm reboots are required.
NIC Mode	DPU Mode	Copy Copied! `curl -k -u root:'password' -H 'content-type: application/json' -d '{"Mode": "DpuMode"}' -X POST https://bmc_ip/redfish/v1/Systems/Bluefield/Oem/Nvidia/Actions/Mode.Set` To apply configuration, two consecutive Arm reboots are required.
DPU Mode	DPU Mode with Zero Trust	Roadmap
DPU Mode with Zero Trust	DPU Mode	Roadmap
DPU Mode with Zero Trust	NIC Mode	Not supported. Move from DPU Mode with Zero Trust to DPU Mode first, and then from DPU Mode to NIC Mode. Warning Perform a system-level reset when moving from Zero Trust to DPU Mode before configuring NIC Mode. Ensure to disable Zero Trust configuration before transitioning to NIC Mode. Operating in NIC Mode with Zero Trust (Restricted) configuration is not supported and may lead to undefined behavior.

Using Platform-BMC NC-SI OEM Commands

From Mode

To Mode

Command

DPU Mode

NIC Mode

Command = 0x12, Parameter = 0x33

Offset	31:24	23:16	15:8	7:0
0:15	NC-SI OEM Header (OEM Command)
16:19	Mellanox Manufacture ID (IANA) = 0x8119
20:23	Command rev=0x00	MLNX Cmd ID= 0x12	Parameter=0x33	Reserved
24:27					Offload engine
28:31	Checksum 31:0

Field

Size

Offset in NC-SI Command

Description

Offload engine

1 bit

27.0

0x0: Enabled – DPU Mode

0x1: Disabled – NIC Mode

For NIC Mode, set the offload engine bit to 0x1.

NIC Mode

DPU Mode

Command = 0x12, Parameter = 0x33

bytes/bits	31:24	23:16	15:8	7:0
0:15	NC-SI OEM Header (OEM Command)
16:19	Mellanox Manufacture ID (IANA) = 0x8119
20:23	Command rev=0x00	MLNX Cmd ID= 0x12	Parameter=0x33	Reserved
24:27					Offload engine
28:31	Checksum 31:0

Field

Size

Offset in NC-SI Command

Description

Offload engine

1 bit

27.0

0x0: Enabled - DPU Mode

0x1: Disabled - NIC Mode

For DPU Mode, set the offload engine bit to 0x0.

DPU Mode

DPU Mode with Zero Trust

Roadmap

DPU Mode with Zero Trust

DPU Mode

Roadmap

DPU Mode with Zero Trust

NIC Mode

Not supported. Move from DPU Mode with Zero Trust to DPU Mode first, and then from DPU Mode to NIC Mode.

Warning

Perform a system-level reset when moving from Zero Trust to DPU Mode before configuring NIC Mode.
Ensure to disable Zero Trust configuration before transitioning to NIC Mode.
Operating in NIC Mode with Zero Trust (Restricted) configuration is not supported and may lead to undefined behavior.

Warning

Separated Host Mode is obsolete for BlueField DPU/SuperNIC SKUs and should not be used, even if it remains visible in some configuration menus or options.

Note

Separated Host Mode is available only for BlueField controller SKUs, where the BlueField Arm OS is the sole CPU/OS in the chassis.

On This Page