Introduction

In PCIe x16 Configuration

PCIe Gen 5.0 (32GT/s) through x16 edge connector.

In PCIe x16 Extension Option - Switch DSP (Data Stream Port)

• PCIe Gen 5.0 SERDES @32GT/s through edge connector

• PCIe Gen 5.0 SERDES @32GT/s through PCIe Auxiliary Connection Card

The maximum power consumption is up to 150W and is split between the two power sources as follows:

• Up to 66W from the PCIe golden fingers (12V)

• The rest of the consumed power is drawn from a supplementary 8-pin ATX power supply connector

The BlueField-3 DPU Controller is shipped with Ubuntu – a Linux commercial operating system – which includes the NVIDIA OFED stack (MLNX_OFED), and is capable of running all customer-based Linux applications seamlessly. For more information, please refer to the DOCA SDK documentation or NVIDIA BlueField-3 Software User Manual.

• Interoperable with 1/10/25/40/50/100/200 Gb/s Ethernet switches and SDR/DDR/EDR/HDR100/HDR/NDR200 InfiniBand switches

• Passive copper cable with ESD protection

• Powered connectors for optical and active cable support

1x BlueField-3 DPU Controller

1x tall bracket (shipped assembled on the card)

MBF35-DKIT

4-pin USB to female USB Type-A cable

20-pin shrouded connector to USB Type-A cable

InfiniBand Architecture Specification v1.5 compliant

The DPU Controller delivers low latency, high bandwidth, and computing efficiency for high-performance computing (HPC), artificial intelligence (AI), and hyperscale cloud data centers applications.

The DPU Controller is InfiniBand Architecture Specification v1.5 compliant.

InfiniBand Network Protocols and Rates:

Up to 200 Gigabit Ethernet

The DPU Controller complies with the following IEEE 802.3 standards:

200GbE / 100GbE / 50GbE / 40GbE / 25GbE / 10GbE

On-board Memory

• Quad SPI NOR FLASH - includes 256Mbit for Firmware image.

• UVPS EEPROM - includes 2Mbit.

• FRU EEPROM - Stores the parameters and personality of the card. The EEPROM capacity is 128Kbit. FRU I2C address is (0x50) and is accessible through the PCIe SMBus.

• DPU_BMC Flashes:

  • 2x 64MByte for BMC Image

  • 512MByte for Config Data

• eMMC pSLC 40GB with 30K Write Cycles eMMC for SoC BIOS and OS.

• SSD (onboard BGA) 128GByte for user SoC OS, logs and application SW.

• DDR5 SDRAM - 48GB @5200MT/s dual-channel DDR5 SDRAM memory. Solder down on-board 128bit + 16 bit ECC.

BlueField-3 SoC

The DPU Controller integrates x16 Armv8.2+ A78 Hercules cores (64-bit) is interconnected by a coherent mesh network, one DRAM controller, an RDMA intelligent network adapter supporting up to 400Gb/s, an embedded PCIe switch with endpoint and root complex functionality, and up to 32 lanes of PCIe Gen 5.0.

RDMA and RDMA over Converged InfiniBand/Ethernet (RoCE)

The DPU Controller, utilizing IBTA RDMA (Remote Data Memory Access) and RoCE (RDMA over Converged InfiniBand/Ethernet) technology, delivers low-latency and high-performance over InfiniBand/Ethernet networks. Leveraging data center bridging (DCB) capabilities as well as advanced congestion control hardware mechanisms, RoCE provides efficient low-latency RDMA services over Layer 2 and Layer 3 networks.

Storage Acceleration

• A consolidated compute and storage network achieves significant cost-performance advantages over multi-fabric networks. Standard block and file access protocols can leverage RDMA for high-performance storage access: NVMe over Fabric offloads for the target machine

• The DPU Controller may operate as a co-processor offloading specific storage tasks from the host, isolating part of the storage media from the host, or enabling abstraction of software-defined storage logic using the NVIDIA BlueField-3 Arm cores. On the storage initiator side, the DPU Controller can prove an efficient solution for hyper-converged systems to enable the host CPU to focus on compute while all the storage interface is handled through the Arm cores.

NVMe-oF and NVMe/TCP Accelerations

Non-volatile Memory Express (NVMe) over Fabrics or over TCP are protocols for communicating block storage IO requests over RDMA or TCP to transfer data between a host computer and a target solid-state storage device or system over a network. The DPU Controller may operate as a co-processor offloading specific storage tasks from the host using its powerful NVMe Offload accelerator.

SR-IOV

The DPU Controller SR-IOV technology provides dedicated adapter resources and guaranteed isolation and protection for virtual machines (VM) within the server.

GPU Direct

GPUDirect RDMA is a technology that provides a direct P2P (Peer-to-Peer) data path between the GPU Memory directly to/from the NVIDIA HCA devices. This provides a significant decrease in GPU-GPU communication latency and completely offloads the CPU, removing it from all GPU-GPU communications across the network. The DPU Controller uses high-speed DMA transfers to copy data between P2P devices resulting in more efficient system applications

Cryptography Accelerations

From IPsec and TLS data-in-motion inline encryption to AES-XTS block-level data-at-rest encryption and public key acceleration, the DPU Controller hardware-based accelerations offload the crypto operations and free up the CPU, reducing latency and enabling scalable crypto solutions. BlueField-3 “host-unaware” solutions may transmit and receive data, while BlueField-3 acts as a bump-in-the-wire for crypto.

Security Accelerators

A consolidated compute and network solution based on the DPU Controller achieves significant advantages over a centralized security server solution. Standard encryption protocols and security applications can leverage NVIDIA BlueField-3 compute capabilities and network offloads for security application solutions such as Layer4 Statefull Firewall.

Out-of-Band Management

The DPU Controller incorporates a 1GbE RJ45 out-of-band port that allows the network operator to establish trust boundaries in accessing the management function to apply it to network resources. It can also be used to ensure management connectivity (including the ability to determine the status of any network component) independent of the status of other in-band network components.

BMC

The DPU Controller incorporate local NIC BMC (Baseboard Management Controller) hardware on the board. The BMC SoC (system on a chip) can utilize either shared or dedicated NICs for remote access. The BMC node enables remote power cycling, board environment monitoring, BlueField-3 chip temperature monitoring, board power and consumption monitoring, and individual interface resets. The BMC also supports the ability to push a bootstream to BlueField-3.

Having a trusted on-board BMC that is fully isolated for the host server ensures highest security for the DPU boards.