Supported Interfaces
This section describes the DPU-supported interfaces. Each numbered interface that is referenced in the figures is described in the following table with a link to detailed information.
The below figures are for illustration purposes only and might not reflect the current revision of the DPU.
Interfaces of MBF2M345A-HECOT, MBF2M345A-HECOT
Component Side |
Print Side |
|
|
Item |
Interface |
Description |
1 |
DPU |
DPU IC 8 cores |
2 |
PCI Express Interface |
PCIe Gen 4.0 through an x16 edge connector |
3 |
Networking Interface |
Network traffic is transmitted through the DPU QSFP56 connector. The QSFP56 connectors allow for the use of modules, and optical and passive cable interconnect solutions. |
4 |
Networking Ports LEDs Interface |
One bi-color LED per port for the link and physical status |
5 |
DDR4 SDRAM On-Board Memory |
8 units of SDRAM for a total of 16GB @ 3200MT/s single DDR4 channel, 64bit + 8bit ECC, solder-down memory |
6 |
NC-SI Management Interface |
Connection for remote sideband management |
7 |
USB 4-pin vertical connector (default) |
Mounted on the DPU for OS image loading |
8 |
1GbE OOB Management Interface |
1GbE BASE-T OOB management interface |
9 |
RTC Battery |
Battery holder for RTC |
10 |
eMMC Interface |
x8 NAND flash |
Interfaces of MBF2H516A-EEEOT, MBF2H516A-EENOT, MBF2M516A-EECOT, MBF2M516A-EEEOT and MBF2M516A-EENOT
Component Side |
Print Side |
|
|
Item |
Interface |
Description |
1 |
DPU |
DPU IC 8 cores |
2 |
PCI Express Interface |
PCIe Gen 4.0 through an x16 edge connector |
3 |
Networking Ports |
The network traffic is transmitted through the DPU QSFP56 connectors. The QSFP56 connectors allow the use of modules, optical and passive cable interconnect solutions. |
4 |
Networking Ports LEDs Interface |
One bi-color I/O LED per port to indicate link and physical status |
5 |
DDR4 SDRAM On-Board Memory |
8 units of SDRAM for a total of 16GB @ 3200MT/s single DDR4 channel, 64bit + 8bit ECC, solder-down memory |
6 |
NC-SI Management Interface |
Connection for remote sideband management |
7 |
Mini USB Type B Interface |
Used for OS image loading |
8 |
1GbE OOB Management Interface |
1GbE BASE-T OOB management interface |
9 |
External PCIe Power Supply Connector |
An external 12V power connection through a 6-pin ATX connector. NOTE: This connector is present on FHHL P-Series DPUs only. It is not present on FHHL E-Series DPUs. |
10 |
RTC Battery |
Battery holder for RTC |
11 |
eMMC Interface |
x8 NAND flash |
Interfaces of MBF2H516C-EECOT, MBF2H516C-EESOT, MBF2M516C-EECOT and MBF2M516C-EESOT
Component Side |
Print Side |
|
|
Item |
Interface |
Description |
1 |
DPU |
DPU IC 8 cores |
2 |
PCI Express Interface |
PCIe Gen 4.0 through an x16 edge connector |
3 |
Networking Ports |
The network traffic is transmitted through the DPU QSFP56 connectors. The QSFP56 connectors allow the use of modules, optical and passive cable interconnect solutions. |
4 |
Networking Ports LEDs Interface |
One bi-color I/O LED per port to indicate link and physical status |
5 |
DDR4 SDRAM On-Board Memory |
8 units of SDRAM for a total of 16GB @ 3200MT/s single DDR4 channel, 64bit + 8bit ECC, solder-down memory |
6 |
NC-SI Management Interface |
Connectivity for remote sideband management (NC-SI over RBT). The NC-SI connector type differs per the product HW version: |
7 |
USB 4-pin vertical connector |
Used for OS image loading |
8 |
1GbE OOB Management Interface |
1GbE BASE-T OOB management interface |
9 |
External PCIe Power Supply Connector |
An external 12V power connection through a 6-pin ATX connector. NOTE: This connector is present on FHHL P-Series DPUs only. It is not present on FHHL E-Series DPUs. |
10 |
RTC Battery |
Battery holder for RTC |
11 |
eMMC Interface |
x8 NAND flash |
12 |
BMC |
Embedded BMC on DPU |
13 |
MMCX RA PPS IN/OUT |
Enables PPS IN/OUT |
DPU IC
NVIDIA® BlueField®-2 DPU is a family of advanced DPU IC solutions that integrate a coherent mesh of 64-bit Arm v8 A72 cores, an NVIDIA® ConnectX®-6 Dx network adapter front-end, and a PCI Express switch into a single chip. The powerful DPU IC architecture includes an Armv8 multicore processor array and enables customers to develop sophisticated applications and highly differentiated feature sets. leverages the rich Arm software ecosystem and introduces the ability to offload the x86 software stack.
At the heart BlueField-2, the ConnectX-6 Dx network offload controller with RDMA and RDMA over Converged Ethernet (RoCE) technology delivers cutting-edge performance for networking and storage applications such as NVMe over Fabrics. Advanced features include an embedded virtual switch with programmable access lists (ACLs), transport offloads, and stateless encaps/decaps of NVGRE, VXLAN, and MPLS overlay protocols.
Encryption
Applies to Crypto enabled OPNs.
DPU addresses the concerns of modern data centers by combining hardware encryption accelerators with embedded software and fully integrated advanced network capabilities, making it an ideal platform for developing proprietary security applications. It enables a distributed security architecture by isolating and protecting each individual workload and providing flexible control and visibility at the server and workload level, controlling risk at the server access layer. builds security into the DNA of the data center and enables prevention, detection, and response to potential threats in real-time. The DPU can deliver powerful functionality, including encryption of data-in-motion, bare-metal provisioning, a stateful L4 firewall, and more.
Networking Ports Interface
The network ports of the DPU are compliant with the InfiniBand and the IEEE 802.3 Ethernet standards listed in Features and Benefits. Traffic is transmitted through the cards' QSFP56 connectors. By default, the port cages of this group of OPNs are set to operate in QSFP28 mode (default card firmware setting). BlueField-2 DPUs support copper/optic and SR4 modules only.
Networking Ports LEDs Interface
There is one bicolor (Yellow and Green) I/O LED per port to indicate speed and link status.
State |
Bi-Color LED (Yellow/Green) |
||||||||||
Beacon command for locating the adapter card |
1Hz blinking Yellow |
|
|||||||||
Error |
4Hz blinking Yellow Indicates an error with the link. The error can be one of the following:
|
|
|||||||||
Physical Activity |
Blinking Green |
|
|||||||||
Link Up |
Solid Green |
|
|||||||||
Physical Up (InfiniBand Mode Only) |
Solid Yellow |
|
PCI Express Interface
The DPU supports PCI Express Gen 4.0 (1.1, 2.0 and 3.0 compatible) through x16 edge connectors. The following lists PCIe interface features:
PCIe Gen 4.0, and 3.0, 2.0 and 1.1 compatible
2.5, 5.0, or 8.0, or 16.0 GT/s link rate x16 lanes
Auto-negotiates to x16, x8, x4, x2, or x1
Support for MSI/MSI-X mechanisms
DDR4 SDRAM On-Board Memory
The DPU incorporates 16GB @ 3200MT/s single DDR4 channel, 64bit + 8bit ECC, solder-down memory.
NC-SI Management Interface
The DPU enables the connection of a Baseboard Management Controller (BMC) to a set of Network Interface Controller (NICs) to enable out-of-band remote manageability. The NC-SI management is supported over RMII and has a connector on the DPU. For connecting to the NCSI RBT interface on the 20 or 30-pin connector, a customized cable is needed based on the NC-SI Management Interface pinouts.
The below table specifies the maximum trace lengths on the board per board type. Please take the maximum trace length on the board into consideration in your design.
The USB to UART cable is not used for NC-SI management purposes.
DPU OPN |
Maximum Trace Length on the Board |
MBF2M516C-EECOT |
5 inch in 30-pin connector |
MBF2M345A-HECOT |
3.3inch |
MBF2H516A-EEEOT |
6inch |
UART Interface Connectivity
A UART debug interface is available on the DPU cards via the NC-SI connector. The below table describes the UART interface location and connectivity per the NC-SI connector type on the DPU you have purchased.
NC-SI Connector Type |
UART Interface Location and Connectivity |
||||||||||||
30-pin |
For DPUs with onboard BMC, the UART interface is that of the BlueField-2 device. For DPUs without onboard BMC, the UART interface is that of the NIC BMC device.
Please note the following: The UART interface is compliant with the TTL 3.3V voltage level. A USB to UART cable that supports TTL voltage levels should be used to connect the UART Interface for Arm console access - see example below. Please refer to UART Cable Installation for installation instructions.
Warnings:
|
||||||||||||
20-pin |
DPUs with onboard BMC hardware: The UART interface is that of the NIC BMC device.
|
USB Interfaces
The USB interface is used to load operating system images. The following table list the types of onboard USB interfaces per DPU part number.
OPN |
USB Interface Type |
USB Cable |
MBF2M345A-HECOT |
USB 4-pin vertical connector |
Use a 4-pin male connector to a male Type-A cable to connect to the board. The cable is not included in the shipped DPU card box and should be ordered separately as part of the accessories kit (P/N: MBF25-DKIT).
|
MBF2H516A-EEEOT |
Mini USB Type-B connector |
Use a standard USB Type-B to connect. |
BMC Interface
Applicable to MBF2H516C-EECOT, MBF2H516C-EESOT, MBF2M516C-EECOT and MBF2M516C-EESOT.
Some DPUs incorporate local NIC BMC (Baseboard Management Controller) hardware on the board. The BMC SoC (system on a chip) can utilize either shared or dedicated NICs for remote access. The BMC node enables remote power cycling, board environment monitoring, BlueField-2 chip temperature monitoring, board power, and consumption monitoring, and individual interface resets. The BMC also supports the ability to push a boot stream to BlueField-2.
Having a trusted onboard BMC that is fully isolated from the host server ensures the highest security for the DPU boards.
For more information, please refer to Connecting to BMC Interfaces.
1GbE OOB Management Interface
The DPU incorporates a 1GbE RJ-45 out-of-band port that allows the network operator to establish trust boundaries in accessing the management function to apply it to network resources. It can also be used to ensure management connectivity (including the ability to determine the status of any network component) independent of the status of other in-band network components.
10Mb/s and 100Mb/s modes are not supported on this interface.
For DPUs with integrated BMC: 1GbE OOB Management can be performed via the BlueField-2 device or the integrated BMC.
1GbE OOB Management LEDs Interface
There are 2 OOB management LEDs, one green and one amber/yellow. The following table describes LED behavior for DPUs with or with onboard BMC.
LED Indications |
Link Activity |
||
Green LED |
Amber/Yellow LED |
DPUs with BMC |
DPUs without BMC |
OFF |
OFF |
Link off |
Link off |
ON |
OFF |
1 Gb/s link / No activity |
Link on (any speed *) / No activity |
Blinking |
OFF |
1 Gb/s link / Activity (RX,TX) |
1 Gb/s link / Activity (RX,TX) |
OFF |
ON |
Not supported |
100 Mb/s link / No activity |
OFF |
Blinking |
100 Mb/s link / Activity (RX,TX) |
|
ON |
ON |
10 Mb/s link / No activity |
|
Blinking |
Blinking |
10 Mb/s link / Activity (RX,TX) |
* On DPUs without BMC, speeds can be 10 Mb/s, 100 Mb/s or 1Gb/s.
External PCIe Power Supply Connector
Applies to FHHL P-Series DPUs only, OPNs MBF2H516A-CEEOT and MBF2H516A-CENOT.
FHHL P-Series DPUs require supplementary power from a 6-pin ATX power supply connector in addition to a minimum of 75W power through the x16 PCIe interface. The power cable should be supplied by the customer.
The FHHL DPUs incorporate an external 12V power connection through a 6-pin ATX connector. The DPU includes a special circuitry that provides current balancing between the two power supplies; the 12V from the PCIe x16 standard slot and the 12V from the ATX 6-pin connector. Since the power provided by the PCIe golden fingers is limited to 75W, a total maximum of up to 150W is enabled through both the ATX 6-pin connector and the PCIe x16 golden fingers. The actual power consumption is in accordance to the mode of operation of the DPU and is split evenly between the two power sources.
For the external PCIe power supply pins, please refer to External PCIe Power Supply Connector Pins.
RTC Battery
The DPU incorporates a COIN TYPE LITHIUM BATTERY CR621 for RTC (Real Time Clock).
eMMC Interface
The DPU incorporates an eMMC interface on the card's print side. The eMMC is an x8 NAND flash and is used for Arm boot, operating system storage and disk space. Memory size is either 64GB or 128GB, where 128GB is effectively 40GB with high durability.
MMCX RA PPS IN/OUT Interface
Applies to the following OPNs: MBF2H516C-EECOT, MBF2H516C-EESOT, MBF2M516C-EECOT, MBF2M516C-EESOT.
The DPU incorporates an integrated Hardware Clock (PHC) to allow the DPU to achieve sub-20u Sec accuracy and also offers many timing-related functions such as time-triggered scheduling or time-based SND accelerations (time-based ASAP²). Furthermore, 5T technology enables the software application to transmit fronthaul (ORAN) at high bandwidth. The PTP part supports the subordinate clock, master clock, and boundary clock.
The DPU PTP solution allows you to run any PTP stack on your host.
With respect to testing and measurements, selected NVIDIA DPUs allow you to use the PPS-out signal from the onboard MMCX RA connectors; the DPU also allows measuring PTP in scale, with the PPS-In signal. The PTP HW clock on the Network adapter will be sampled on each PPS-In signal, and the timestamp will be sent to the SW.