This is the User Manual for BlueField® BF1600 Controller Card. This document contains the complete installation instructions and specifications for the BF1600 Controller Cards listed in the below table.
Ordering Part Numbers
The below table provides the ordering part numbers (OPN) for the available BlueField BF1600 Controller Cards:
| OPN | Marketing Description |
| MBF1M606A-CSNAT | BlueField® Controller card, Dual Port 100Gb/s QSFP28, PCIe Gen4.0 x16, BlueField® E-Series 16 cores, Crypto disabled, No Memory DIMM, FH¾L, Single Slot, Tall Bracket |
| MBF1M606A-ESNAT | BlueField® Controller card, Dual Port InfiniBand/Ethernet 100Gb/s QSFP28, PCIe Gen4.0 x16, BlueField® E-Series 16 cores, Crypto disabled, No Memory DIMM, FH¾L, Single Slot, Tall Bracket |
| MBF1M616A-CSNAT | BlueField® Controller card, Dual Port 100Gb/s QSFP28, PCIe Gen4.0 x16, BlueField® E-Series 16 cores, Crypto disabled, 2x 16GB Memory DIMM, FH¾L, Single Slot, Tall Bracket |
| MBF1M626A-CSNAT | BlueField® Controller card, Dual Port 100Gb/s QSFP28, BlueField E-Series 16 cores, PCIe Gen3.0 x16, Auxiliary Card PCIe Gen3.0 x16, CABLINE-CA II PLUS 350mm, Crypto disabled, No Memory DIMM, FH¾L, Single Slot width, Tall Bracket |
| MBF1M636A-CSNAT | BlueField® Controller card, Dual Port 100Gb/s QSFP28, BlueField E-Series 16 cores, PCIe Gen3.0 x16, Auxiliary Card PCIe Gen3.0 x16, CABLINE-CA II PLUS 350mm, QSFP28, 2x 16GB Memory DIMM, FH¾L, Single Slot width, Tall Bracket |
| MBF1M646A-CSNAT | BlueField® Controller card, Dual Port 100Gb/s, 100GbE dual-port QSFP28, PCIe Gen4.0 x16, Crypto disabled, 2x8GB Memory DIMM, FH¾L, Single Slot, Tall Bracket |
| MBF1M656A-CSNAT | BlueField® Controller card, Dual Port 100Gb/s QSFP28, BlueField® E-Series 16 cores, PCIe Gen3.0 x16, Auxiliary Card PCIe Gen3.0 x16, 2 x I-PEX 350mm Cable, Crypto disabled, 2x 8GB Memory DIMM, FH¾L, Single Slot, Tall Bracket |
Product Overview
The BlueField BF1600 Dual Port 100Gb/s Controller Card is the perfect solution for managing NVMe storage drives in storage and hyper-converged systems. BlueField BF1600 Dual Port 100Gb/s Controller Card delivers the highest NVMe-oF target performance, reducing TCO and increasing ROI. The BlueField BF1600 Controller Cards can transform existing JBOF systems into NVMe-oF compliant solutions, simply by plugging the card into an existing PCIe slot. The card is available in two configurations: PCIe x16 and PCIe x32. This flexibility makes it easier to match different JBOF system configurations. The card’s small form factor allows customers to install multiple controller cards in a single system to support a larger number of SSDs and high-availability storage architecture.
The BlueField BF1600 Controller Card should be installed only in a JBOF System as it functions as a PCIe root-complex (RC) initiating PCIe bus operations. Installing it in a regular host system may damage the card.
BlueField Multicore System-on-chip (SoC)
BlueField SoC is a highly integrated and efficient controller, optimized for NVMe storage systems, Network Functions Virtualization (NFV), Cloud and AI workloads.
BlueField SoC integrates all the discrete components of a storage system appliance into a single chip (CPU, PCIe switch, and Network controller), making it the premier SoC solution for building Just-a-Bunch-Of-Flash (JBOF) systems, All-Flash-Array and storage appliances for NVMe over-Fabrics.
With an integrated NVMe-oF offload accelerator, the BlueField SoC gives the BlueField Dual Port 100Gb/s Controller Card a superior performance advantage over existing JBOF systems, significantly reducing storage transaction latency, while increasing IOPs (I/O operations per second).
BlueField SoC also offers cost-effective and integrative solutions for Machine Learning appliances, enabling efficient data delivery for real-time analytics and data insights based on superior RDMA and GPUDirect® RDMA technologies. Support for PCIe Gen4.0 helps future-proof the next generation of high-performance GPU cards.
BlueField Controller Cards
| Part Number | MBF1M606A-CSNAT | MBF1M646A-CSNAT | MBF1M616A-CSNAT | MBF1M626A-CSNAT | MBF1M656A-CSNAT | MBF1M636A-CSNAT |
| Form Factor | PCIe Full Height, ¾ Length 9.13 in. x 4.37 in. (232.0 mm x 111.15mm) | |||||
| Heatsink Height | 10.1mm | |||||
| Ethernet Data Rate | 1/10/25/40/50/100 Gb/s Ethernet | |||||
| Network Connector Type | Dual-port QSFP28 | |||||
| PCIe Gen 3.0/4.0 through x16 Edge Connectora | PCIe Gen 3.0 / 4.0 SERDES @ 8.0GT/s / 16.0GT/s | |||||
| PCIe Gen 3.0 through x16 Auxiliary Card | NA | PCIe Gen 3.0 SERDES @ 8.0GT/s | ||||
On-board DDR4 Memory(a) | NA | 2x 8GB | 2x 16GB | NA | 2x 8GB | 2x 16GB |
| 1GbE OOB Management | √ | √ | √ | √ | ||
| RoHS | RoHS Compliant | |||||
BlueField SoC | BlueField E-Series 16 cores | |||||
a. Note: Refer to BlueField Software and Firmware release notes for the availability of PCIe Gen 4.0 capabilities.
For more detailed information see Specifications.
Features and Benefits
This section describes hardware features and capabilities. Please refer to the software release notes for feature availability.
It is recommended to upgrade your BlueField product to the latest software and firmware versions available in order to enjoy the latest features and bug fixes.
| Feature | Description |
|---|---|
| PCI Express (PCIe)a | Uses the following PCIe connectors: |
| Up to 100 Gigabit Ethernet | • 100GbE/ 50GbE / 25GbE |
| On-board Memory | • Two DDR4 SODIMM sockets - size is 16GB each (MBF1M636A-CSNAT and MBF1M616A-CSNAT) • Two DDR4 SODIMM sockets - size is 8GB each (MBF1M656A-CSNAT and MBF1M646A-CSNAT) |
| BlueField SoC | The BlueField-2 SoC integrates eight 64-bit Armv8 A72 cores interconnected by a coherent mesh network, one DRAM controller, an RDMA intelligent network adapter supporting up to 200Gb/s, an embedded PCIe switch with endpoint and root complex functionality, and up to 16 lanes of PCIe Gen 3.0/4.0. |
| Overlay Networks | In order to better scale their networks, data center operators often create overlay networks that carry traffic from individual virtual machines over logical tunnels in encapsulated formats such as NVGRE and VXLAN. While this solves network scalability issues, it hides the TCP packet from the hardware offloading engines, placing higher loads on the host CPU. BlueField BF1600 Controller Card effectively addresses this by providing advanced NVGRE and VXLAN hardware offloading engines that encapsulate and de-capsulate the overlay protocol. |
| RDMA and RDMA over Converged Ethernet (RoCE) | The BlueField BF1600 Controller Cards, utilizing IBTA RDMA (Remote Data Memory Access) and RoCE (RDMA over Converged Ethernet) technology, deliver low-latency and high-performance over Ethernet networks. Leveraging data center bridging (DCB) capabilities as well as advanced congestion control hardware mechanisms, RoCE provides efficient low-latency RDMA services over Layer 2 and Layer 3 networks. |
| NVIDIA PeerDirect® | PeerDirect communication provides high-efficiency RDMA access by eliminating unnecessary internal data copies between components on the PCIe bus (for example, from GPU to CPU), and therefore significantly reduces application run time. BlueField BF1600 Controller Card advanced acceleration technology enables higher cluster efficiency and scalability to tens of thousands of nodes. |
| Quality of Service (QoS) | Support for port-based Quality of Service enabling various application requirements for latency and SLA. |
| Storage Acceleration | A consolidated compute and storage network achieves significant cost-performance advantages over multi-fabric networks. Standard block and file access protocols can leverage RDMA for high-performance storage access. • NVMe over Fabric offloads for the target machine BlueField-2 SmartNIC may operate as a co-processor offloading specific storage tasks from the host, isolating part of the storage media from the host, or enabling abstraction of software-defined storage logic |
| High-Performance Accelerations |
|
| GPU Direct | The latest advancement in GPU-GPU communications is GPUDirect RDMA. This new technology provides a direct P2P (Peer-to-Peer) data path between the GPU Memory directly to/from the NVIDIA HCA devices. This provides a significant decrease in GPU-GPU communication latency and completely offloads the CPU, removing it from all GPU-GPU communications across the network. BlueField BF1600 Controller Card uses high-speed DMA transfers to copy data between P2P devices resulting in more efficient system applications. |
| Security Accelerators | A consolidated compute and network solution based on BlueField Controller Card achieves significant advantages over a centralized security server solution. Standard encryption protocols and security applications can leverage BlueField compute capabilities and network offloads for security application solutions such as: |
a. Refer to BlueField Software and Firmware release notes for availability of PCIe Gen 4.0 capabilities
Operating Systems/Distributions
The BlueField® BF1600 Controller Card supports the following operating systems:
- BlueField Software
- OpenFabrics Enterprise Distribution (OFED)
Connectivity
- Interoperable with 1/10/25/40/50/100 Gb/s Ethernet switches
- Passive copper cable with ESD protection
- Powered connectors for optical and active cable support
Package Contents
Before installing your new system, unpack it and check against the below tables that all the parts have been sent according to the OPN you purchased. Check the parts for visible damage that may have occurred during shipping.
If anything is damaged or missing, contact your sales representative at support@NVIDIA.com.
BF1600 Controller Card with Auxiliary Connection Card
Applies to MBF1M626A-CSNAT, MBF1M656A-CSNAT and MBF1M636A-CSNAT.
| Category | Qty. | Item |
|---|---|---|
| Cards | 1 | BF1600 Controller Card |
| 1 | PCIe Auxiliary Card | |
| Cables | 1 | USB 2.0 Cable Type A to Type B |
| 1 | USB 2.0 Cable to UART | |
| 2 | CABLINE-CAII PLUS 60 PIN harnesses (white and black) - 35cm | |
| 1 | Tall bracket for the BF1600 Controller Card (shipped assembled) | |
| Accessories | 1 | Tall bracket for the PCIe Auxiliary card (shipped assembled) |
| 1 | Short bracket for the PCIe Auxiliary card |
BF1600 Controller Card without Auxiliary Connection Card
Applies to MBF1M606A-CSNAT, MBF1M646A-CSNAT and MBF1M616A-CSNAT.
| Category | Qty. | Item |
|---|---|---|
| Cards | 1 | BF1600 Controller Card |
| Cables | 1 | USB 2.0 Cable Type A to Type B |
| 1 | USB 2.0 Cable to UART | |
| Accessories | 1 | Tall bracket for the PCIe Auxiliary card (shipped assembled) |
