Introduction

NVIDIA BlueField BF1500 Ethernet Controller Card User Guide

The BlueField BF1500 Controller Card is the perfect solution for managing NVMe storage drives in storage and hyper-converged systems. The BF1500 Controller Card delivers the highest NVMe-oF target performance, reducing TCO and increasing ROI. It can transform existing JBOF systems into NVMe-oF compliant solutions, simply by plugging the card into an existing PCIe slot. The card’s small form factor allows customers to install multiple controller cards in a single system to support a larger number of SSDs and high-availability storage architecture.

Warning

The BF1500 Controller card connectivity to NVMe SSD devices is possible through the PCIe switch.

Warning

The BlueField Controller Card should be installed only in a JBOF System as it functions as a PCIe root-complex (RC) initiating PCIe bus operations. Installing it in a regular host system may damage the card.

BlueField Multicore System-on-chip (SoC)

BlueField SoC is a highly integrated and efficient controller, optimized for NVMe storage systems, Network Functions Virtualization (NFV), Cloud, and AI workloads. BlueField SoC integrates all the discrete components of a storage system appliance into a single chip (CPU, PCIe switch, and Network controller), making it the premier SoC solution for building Just-a-Bunch-Of-Flash (JBOF) systems, All-Flash-Array, and storage appliances for NVMe over-Fabrics.
With an integrated NVMe-oF offload accelerator, the BlueField SoC gives the BF1500 Controller Card a superior performance advantage over existing JBOF systems, significantly reducing storage transaction latency, while increasing IOPs (I/O operations per second).
BlueField SoC also offers cost-effective and integrative solutions for Machine Learning appliances, enabling efficient data delivery for real-time analytics and data insights based on superior RDMA and GPUDirect® RDMA technologies. Support for PCIe Gen4.0 helps future-proof the next generation of high-performance GPU cards.

BlueField BF1500 Ethernet Controller Card Card

Part Number

MBF1L516A-CSNAT

Form Factor

Full-Height, Half-Length (FHHL)

Ethernet Data Rate

1/10/25/40/50/100 GbE

Network Connector Type

Dual-port QSFP28

PCI Express Connectors

PCIe Gen 3.0 (8GT/s) / Gen 4.0 (16GT/s) x16 lanes

On-board DDR4 Memory(a)

Single-channel with 8 DDR4 8 bit + ECC (64bit + 8bit ECC)
16GB @ 2400MT/s

BlueField IC OPN

MT41M16E23A0-CDCR-TTEV

RoHS

RoHS Compliant

Crypto

Disabled

BlueField IC Cores
Series and NIC/Arm
Frequency

16 Cores
G-Series - 275/800MHz

For more detailed information see Specifications.

Warning

This section describes hardware features and capabilities. Please refer to the relevant driver and/or firmware release notes for feature availability.

Feature

Description

PCI Express (PCIe)

Uses PCIe Gen 3.0 (8GT/s) and Gen 4.0 (16GT/s) through an x16 edge connector. Gen 1.1 and 2.0 compatible.

Up to 100 Gigabit Ethernet

NVIDIA Controllers comply with the following IEEE 802.3 standards:
– 100GbE / 50GbE / 40GbE / 25GbE / 10GbE / 1GbE
– IEEE 802.3bj, 802.3bm 100 Gigabit Ethernet
– IEEE 802.3by, Ethernet Consortium25, 50 Gigabit Ethernet, supporting all FEC modes
– IEEE 802.3ba 40 Gigabit Ethernet
– IEEE 802.3by 25 Gigabit Ethernet
– IEEE 802.3ae 10 Gigabit Ethernet
– IEEE 802.3az Energy Efficient Ethernet
– IEEE 802.3ap based auto-negotiation and KR startups
– Proprietary Ethernet protocols (20BASE-R2)
– IEEE 802.3ad, 802.1AX Link Aggregation
– IEEE 802.1Q, 802.1P VLAN tags and priority
– IEEE 802.1Qau (QCN)
– Congestion Notification
– IEEE 802.1Qaz (ETS)
– IEEE 802.1Qbb (PFC)
– IEEE 802.1Qbg
– IEEE 1588v2
– Jumbo frame support (9.6KB)

On-board Memory

  • Quad SPI NOR FLASH - includes 128Mbit for Firmware image (W25Q128FVSIG device by Winbond-Nuvoton).

  • UVPS EEPROM - includes 512Kbit (M24512-DFMC6TG by STMICROELECTRONICS).

  • FRU EEPROM - includes a 128Kbit (M24128-DFMC6TG by STMICROELECTRONICS)

  • eMMC - x8 16GB NAND flash Memory for SoC BIOS.

  • DDR4 SDRAM - single-channel DDR4 SDRAM memory with 8 components of 8 bit + ECC (64bit + 8bit ECC), 8/16GB @2400MT/s.

BlueField SoC

The BlueField SoC integrates four, eight or sixteen 64-bit Armv8 A72 cores interconnected by a coherent mesh network, one DRAM controller, a dual port RDMA intelligent Ethernet adapter supporting up to 100Gb/s, an embedded PCIe switch with endpoint and root complex functionality, and up to 8 lanes of PCIe Gen 3.0/4.0.

Overlay Networks

In order to better scale their networks, data center operators often create overlay networks that carry traffic from individual virtual machines over logical tunnels in encapsulated formats such as NVGRE and VXLAN. While this solves network scalability issues, it hides the TCP packet from the hardware offloading engines, placing higher loads on the host CPU. NVIDIA BF1500 Controller Card effectively addresses this by providing advanced NVGRE and VXLAN hardware offloading engines that encapsulate and de-capsulate the overlay protocol.otocol.

RDMA and RDMA over Converged Ethernet (RoCE)

NVIDIA BF1500 Controller Card, utilizing IBTA RDMA (Remote Data Memory Access) and RoCE (RDMA over Converged Ethernet) technology, delivers low-latency and high-performance over Ethernet networks. Leveraging data center bridging (DCB) capabilities as well as advanced congestion control hardware mechanisms, RoCE provides efficient low-latency RDMA services over Layer 2 and Layer 3 networks.

NVIDIA PeerDirect™

PeerDirect™ communication provides high-efficiency RDMA access by eliminating unnecessary internal data copies between components on the PCIe bus (for example, from GPU to CPU), and therefore significantly reduces application run time. NVIDIA BF1500 Controller Cardadvanced acceleration technology enables higher cluster efficiency and scalability to tens of thousands of nodes.

Quality of Service (QoS)

Support for port-based Quality of Service enabling various application requirements for latency and SLA.

Storage Acceleration

A consolidated compute and storage network achieves significant cost-performance advantages over multi-fabric networks. Standard block and file access protocols can leverage RDMA for high-performance storage access.

  • NVMe over Fabric offloads for the target machine

  • T10-DIF Signature Handover

BlueField BF1500 Controller Card may operate as a co-processor offloading specific storage tasks from the host, isolating part of the storage media from the host, or enabling abstraction of software-defined storage logic
using the BlueField Arm cores. On the storage initiator side, BlueField BF1500 card can prove an efficient solution for hyper-converged systems to enable the host CPU to focus
on compute while all the storage interface is handled through the Arm cores.

NVMe-oF

Nonvolatile Memory Express (NVMe) over Fabrics is a protocol for communicating block storage IO requests over RDMA to transfer data between a host computer and a target solid-state storage device or system over a network. BlueField BF1500 Controller Card may operate as a co-processor offloading specific storage tasks from the host using its powerful NVMe over Fabrics Offload accelerator.

SR-IOV

NVIDIA BF1500 Controller Card SR-IOV technology provides dedicated adapter resources and guaranteed isolation and protection for virtual machines (VM) within the server.

High-Performance Accelerations

  • Tag Matching and Rendezvous Offloads

  • Adaptive Routing on Reliable Transport

  • Burst Buffer Offloads for Background Checkpointing

GPU Direct

The latest advancement in GPU-GPU communications is GPUDirect RDMA. This new technology provides a direct P2P (Peer-to-Peer) data path between the GPU Memory directly to/from the NVIDIA HCA devices. This provides a significant decrease in GPU-GPU communication latency and completely offloads the CPU, removing it from all GPU-GPU communications across the network. NVIDIA BF1500 Controller Card uses high-speed DMA transfers to copy data between P2P devices resulting in more efficient system applications

Security Accelerators

A consolidated compute and network solution based on BF1500 Controller Card achieves significant advantages over a centralized security server solution. Standard encryption protocols and security applications can leverage BlueField compute capabilities and network offloads for security application solutions such as Layer4 Statefull Firewall.

The BF1500 Controller Card is shipped with Linux based Operating System burned on it which includes all needed drivers. For more information, please refer to the BlueField Software User Manual.

  • Interoperable with 1/10/25/40/50/100 Gb/s Ethernet switches

  • Passive copper cable with ESD protection

  • Powered connectors for optical and active cable support

Category

Quantity

Item

Cards

1

BF1500 Controller Card

Cables

1

USB 2.0 Type A to Mini USB Type B cable

1

3 pin UART to USB cable

Accessories

1

Tall bracket (shipped assembled)

1

Short bracket

© Copyright 2023, NVIDIA. Last updated on May 22, 2023.