NVIDIA BlueField-3 Networking Platform User Guide

PCIe Auxiliary Card Kit

Note

This section applies to the following DPUs:

  • 900-9D3B6-00CN-AB0

  • 900-9D3B6-00SN-AB0

  • 900-9D3B6-00CV-AA0

  • 900-9D3B6-00SV-AA0

There are two available PCIe auxiliary kit functionalities:

  1. Utilizing the Socket-Direct capability, where the PCIe extension card is connected to the BlueField-3 DPU, used as an end-point.

  2. Utilizing the Down Stream Port (DSP) extension option, where the PCIe extension card is connected to the BlueField-3 DPU, used as a root complex for storage devices.

The two Cabline SA-II Plus harnesses in the PCIe auxiliary kit have different routings. To distinguish between these two harnesses, one black harness is marked with a “WHITE” label while the harness is marked with a “BLACK” label.

The Cabline harness marked with the "WHITE" label should be connected to the connector on the networking card and PCIe Auxiliary card engraved with “White Cable” while the one marked with the "BLACK" label should be connected to the connector on the networking card and the PCIe Auxiliary card engraved with “Black Cable”. The Cabline SA-II Plus harness mates with two 60 -pin connectors (P/N 20790-060E-01), on both sides. The black Cabline SA-II Plus harness mates with the connector on the component side (top side) of the network card, while the White Cabline SA-II Plus harness mates with the print side (bottom side) of the main network card. For hardware installation, please refer to [Optional] PCIe Extension Connection (2x PCIe x16) Installation Instructions.

Warning

The socket-direct functionality is currently not supported by firmware. Please approach your sales representatives.

Socket Direct network cards, which cost-effectively integrate a single network adapter silicon on a primary board, and an auxiliary PCIe connection card and Cabline SA-II Plus Harness connecting the two. Socket Direct enables direct access from each CPU to the network through its dedicated PCIe interface as the card's 32-lane PCIe bus is split into two 16-lane buses, with one bus accessible through a PCIe x16 edge connector and the other bus through an x16 Auxiliary PCIe Connection card. The two cards should be installed into two PCIe x16 slots and connected using two Cabline SA-II Plus harnesses.

The PCIe auxiliary kit can be purchased separately to operate in a dual-socket server. The table below lists the available PCIe auxiliary kit ordering part numbers, depending on the desired length of the Cabline SA-II Plus harnesses and the PCI Express interface, Gen 4.0 or Gen 5.0.

Ordering Part Number

Passive Auxiliary Connection

Cabline SA-II Plus Harnesses Length

MTMK9100-T15

PCIe Gen 5.0 x16 connection card

2x 150mm harnesses

MTMK9100-T25

PCIe Gen 4.0 x16 connection card

2x 250mm harnesses

MTMK9100-T35

PCIe Gen 4.0 x16 connection card

2x 350mm harnesses

The BlueField-3 DPUs with downstream port extension option integrate a single network adapter silicon on a primary board, and an auxiliary PCIe connection card and Cabline SA-II Plus Harness connecting the two.

This enables direct access from each CPU to the network through its dedicated PCIe interface as the card's 32-lane PCIe bus is split into two 16-lane buses, with one bus accessible through a PCIe x16 edge connector and the other bus through an x16 Auxiliary PCIe Connection card. The two cards should be installed into two PCIe x16 slots and connected using two Cabline SA-II Plus harnesses.

The PCIe auxiliary kit can be purchased separately to allow downstream port extension. The table below lists the available PCIe auxiliary kit ordering part numbers, depending on the desired length of the Cabline SA-II Plus harnesses and the PCI Express interface, Gen 4.0 or Gen 5.0.

OPN

Description

930-9DAX5-0015-000

NVIDIA BlueField-3 Auxiliary Kit for additional self-hosted PCIe Gen 5.0/4.0 x16 connection, two 150mm Cabline SA-II Plus Harnesses

930-9DAX4-0035-000

NVIDIA BlueField-3 Auxiliary Kit for additional self-hosted PCIe Gen 4.0 x16 connection, two 350mm Cabline SA-II Plus Harnesses

930-9DHAR-0001-000

NVIDIA BlueField-3 150mm Gen 5.0 FRU harness, Cabline SA-II Plus 60-pin micro coax 38AWG

PCIe Auxiliary Card Package Contents

Category

Qty

Item

Cards

1

For Socket Direct:

MTMK9100-T15: PCIe x16 Gen 5.0 Auxiliary Connection Card

MTMK9100-T25 and MTMK9100-T35: PCIe x16 Gen 4.0 Auxiliary Connection Card

For Down Stream Port (DSP):

930-9DAX5-0015-000: PCIe x16 Gen 5.0/4.0 Auxiliary Connection Card

930-9DAX4-0035-000: PCIe x16 Gen 4.0 Auxiliary Connection Card

Harnesses

1

Cabline CA-II Plus harness (white) - Length according to kit OPN

1

Cabline CA-II Plus harness (black) - Length according to kit OPN

Accessories

2

Retention Clip for Cabeline harness (shipped assembled on the harnesses - optional)

1

PCIe Auxiliary card short bracket

1

PCIe Auxiliary card tall bracket (shipped assembled on the Auxiliary card)

© Copyright 2024, NVIDIA. Last updated on Oct 8, 2024.