NVIDIA BlueField-2 BF2500 InfiniBand/Ethernet DPU Controller User Manual
NVIDIA BlueField-2 BF2500 InfiniBand_Ethernet DPU Controller User Manual

Supported Interfaces

The BF2500 DPU Controller is a FHHL card. The component height on the top and bottom of the card complies with the PCIe specification. The below figures show the connector and LED designations.

The below figures are for illustration purposes only and might not reflect the current revision of the BF2500 card.

BF2500 DPU Controller - Component Side

image2020-12-10_18-22-32.png

BF2500 DPU Controller - Print Side

image2020-10-13_10-16-30.png

Callout

Interface

Description

1

.Supported Interfaces v1.0#BlueField-2 Data Processing Unit (DPU)

BlueField-2 DPU with heatsink

2

.Supported Interfaces v1.0#PCIe x16 Gen 3.0/4.0 Edge Connector

The interface between the BlueField-2 Controller and the JBOF system.

3

.Supported Interfaces v1.0#InfiniBand QSFP56 Interfaces

The network ports of the ConnectX®-6 adapter cards are compliant with the InfiniBand Architecture Specification, Release 1.3. InfiniBand traffic is transmitted through the cards' QSFP56 connectors.

.Supported Interfaces v1.0#Ethernet QSFP56 Interfaces

Ethernet traffic is transmitted through the controller’s QSFP56 connectors. The QSFP56 connectors allow for the use of Optical modules and cable interconnect solutions.

4

.Supported Interfaces v1.0#DDR4 SODIMM Memory

Single-channel with 8 DDR4 8 bit + ECC (64bit + 8bit ECC) 16GB @ 3200MT/s

5

.Supported Interfaces v1.0#NC-SI Management Interface

Connectivity channel between the BMC CPU and the BlueField-2 Arm Array

6

.Supported Interfaces v1.0#USB Connector

USB Type-B Bracket mounted on card

7

.Supported Interfaces v1.0#1GbE OOB Management Interface

1GbE BASE-T OOB management interface.

8

.Supported Interfaces v1.0#External PCIe Power Supply Interfaces

Along with the PCIe interface, this connector provides power to the board.

9

.Supported Interfaces v1.0#Networking LED Interfaces

One bi-color LED per port.

10

RTC Battery

Battery for RTC.

11

eMMC Interface

16GbE x8 NAND flash10

BlueField-2 Data Processing Unit (DPU)

BlueField-2 is a family of advanced DPU solutions that integrate a coherent mesh of 64-bit Arm v8 A72 cores, a ConnectX network adapter front-end and a PCI Express switch into a single chip. The powerful DPU architecture includes an Armv8 multicore processor array and enables customers to develop sophisticated applications and highly differentiated feature sets. BlueField-2 leverages the rich Arm software ecosystem and introduces the ability to offload the host server software stack.

At the heart of BlueField-2 is the ConnectX-6 Dx network offload controller with RDMA and RDMA over Converged Ethernet (RoCE) technology, delivering cutting-edge performance for networking and storage applications such as NVMe over Fabrics. Advanced features include an embedded virtual switch with programmable access lists (ACLs), transport offloads and stateless encaps/decaps of NVGRE, VXLAN, and MPLS overlay.

Encryption

Warning

Applies to Crypto enabled OPNs.

BlueField-2 BF2500 DPU Controller addresses the concerns of modern data centers by combining hardware encryption accelerators with embedded software and fully integrated advanced network capabilities, making it an ideal platform for developing proprietary security applications. It enables a distributed security architecture by isolating and protecting each individual workload and providing flexible control and visibility at the server and workload level, controlling risk at the server access layer. BlueField-2 builds security into the DNA of the data center and enables prevention, detection, and response to potential threats in real-time. BlueField-2 DPU Controller can deliver powerful functionality, including encryption of data-in-motion, and data-at-rest (AES-XTS) bare-metal provisioning, stateful L4 firewall and more.

InfiniBand QSFP56 Interface

The network ports of the BlueField-2 DPU Controllers are compliant with the InfiniBand Architecture Specification, Release 1.3. InfiniBand traffic is transmitted through the cards' QSFP56 connectors.

Ethernet QSFP56 Interfaces

The network ports of the BlueField-2 BF2500 DPU Controller are compliant with the IEEE 802.3 Ethernet standards listed in Ethernet traffic is transmitted through the cards' QSFP56 connectors.

QSFP56 Port Over-current Protection

The power consumption for each QSFP56 module is controlled by a 5W current limiting power distribution switch. Every power limiting switch is controlled by a signal, which is enabled by default. This signal is originated from the thermal shutdown mechanism. When the output load exceeds the current limit threshold, the power switch limits the current to a safe level.

PCI Express Interfaces

The PCI Express bus is a high-speed interface used to connect the DPU Controller to other peripheral devices (the BMC, PCIe switch or directly to the NVMe SSDs backplane), and operates as a Root-Complex (RC) initiating PCIe bus operations. The BlueField-2 BF2500 DPU Controller supports PCI Express Gen 3.0/4.0 (1.1 and 2.0 compatible) through x16 edge connector which serves as the main interface of the card. This connector enables the connectivity of the system to the BlueField-2 PCIe interface.

The following lists PCIe interface features:

  • PCIe Gen 4.0 and 3.0 compliant, 2.0 and 1.1 compatible

  • 2.5, 5.0, or 8.0, or 16.0 GT/s link rate x16

  • Auto-negotiates to x8, x4, x2, or x1

  • Support for MSI/MSI-X mechanisms

DDR4 Memory

The BlueField-2 BF2500 DPU Controller incorporates one on-board DDR4 memory sized 16GB, 64-bit data and 8-bit Error Correcting Code (ECC) memory.

The following lists the supported DDR4 data rates:

  • 1600 Mb/s

  • 1866 Mb/s

  • 2133 Mb/s

  • 2400 Mb/s

  • 2666 Mb/s

  • 2933 Mb/s

  • 3200 Mb/s

GbE OOB Management

The BlueField-2 BF2500 DPU Controller incorporates a 1GbE RJ45 out-of-band port that allows the network operator to establish trust boundaries in accessing the management function to apply it to network resources. It can also be used to ensure management connectivity (including the ability to determine the status of any network component) independent of the status of other in-band network components.

1GbE OOB Management LEDs Interface

There are two I/O LEDs, LED1 is green and LED2 is Amber, to indicate link activity as described in the below table.

LED Pin

LED Definition

Link Activity

LED2 (Amber)

OFF

Link OFF

ON

Link ON (Any speed)

LED1 (Green)

OFF

No Activity

Blinking

Activity (RX, Tx)

NC-SI Management Interface

The Network Controller Sideband Interface (NC-SI) enables the connection of a Baseboard Management Controller (BMC) to a card for the purpose of enabling out-of-band remote manageability. NC-SI over the RMII interface is routed through the RMII connector with a flat cable.

The UART0 debug interface is connected to the NC-SI connector through the following pins:

  • Pin # 30 - UART0 RX

  • Pin # 28 - UART0 TX

  • Pin # 26 - GND

The UART interface is complaint with TTL 3.3V voltage level.

UART Interface Connectivity

The USB to UART cable that supports TTL voltage levels can be used to connect the UART Interface for Arm console access. See an example of TTL 3.3V voltage levels cable in the below figure.
Please note that the USB to UART cable is not used for NC-SI management purposes.

It is prohibited to directly connect any RS-232 cable! Only TTL 3.3V voltage level cables are supported!

USB to UART Cable - Example

image2019-2-4_13-56-50.png

Mini USB Type B Connector

The BlueField-2 BF2500 DPU Controller uses a Mini USB 2.0 Type B device. The purpose of the USB connector is debugging and loading new versions of the operating systems and firmware.

image2020-3-4_14-57-37.png


External PCIe Power Supply Interfaces

The BlueField-2 BF2500 DPU Controller consumes power from two power sources; an external +12V through an 6-pin ATX connector (reference #9) and an additional +12V through the PCIe x16 edge connector. The BF2500 DPU Controller’s special circuitry balances the power consumption between these two power sources. Each power source must be capable of supplying up to 75W of power. The 3.3V AUX from the card’s PCIe edge connector is used for logics on the BF2500 DPU Controller. This logic includes power for buffering logic for signals to the card’s edge connector, power for the FRU EEPROM, and miscellaneous items. The 3.3V from PCIe edge connector is floating on the board. For the external PCIe power supply pins, please refer to External Power Supply Interfaces.

Warning

The use of an external power supply interface is restricted to the ATX 6-pin PCIe power connector only.


eMMC Interface

The BlueField-2 BF2500 DPU Controller incorporates an eMMC interface on the card's print side. The eMMC is 16GB x8 NAND flash and is used for Arm boot, operating system storage and disk space.

Networking LED Interfaces

There is a one bi-color I/O LED per port to indicate link status. See the below figure for the networking LEDs locations on the BF2500 DPU Controller bracket.

LED1 and LED2 Link Status Indications - Ethernet Protocol:

LED Color and State

Description

Off

A link has not been established

Beacon command for locating the adapter card

1Hz blinking Yellow

Error

4Hz blinking Yellow Indicates an error with the link. The error can be one of the following:

Error Type

Description

LED Behavior

I2C

I2C access to the networking ports fails

Blinks until error is fixed

Over-current

Over-current condition of the networking ports

Blinks until error is fixed

Solid green

Indicates a valid link with no active traffic

Blinking green

Indicates a valid link with active traffic

LED1 and LED2 Link Status Indications - InfiniBand Protocol:

LED Color and State

Description

Off

A link has not been established

Beacon command for locating the adapter card

1Hz blinking Yellow

Error

4Hz blinking Yellow Indicates an error with the link. The error can be one of the following:

Error Type

Description

LED Behavior

I2C

I2C access to the networking ports fails

Blinks until error is fixed

Over-current

Over-current condition of the networking ports

Blinks until error is fixed

Solid amber

Indicates an active link

Solid green

Indicates a valid (data activity) link with no active traffic

Blinking green

Indicates a valid link with active traffic

RTC Battery

The BF2500 DPU Controller incorporates a COIN TYPE LITHIUM BATTERY CR621 for RTC (Real Time Clock).

The BlueField-2 DPU incorporates an eMMC interface on the card's print side. The eMMC is 16GbE x8 NAND flash and is used for Arm boot, operating system storage and disk space.

© Copyright 2023, NVIDIA. Last updated on Sep 11, 2023.