Supported Interfaces
This section describes the ConnectX-8 supported interfaces. Each numbered interface that is referenced in the figures is described in the following table with a link to detailed information.
The below figures show the component side of the NVIDIA ConnectX-8 SuperNIC. Each numbered interface that is referenced in the figures is described in the following table with a link to detailed information.
The below figures are for illustration purposes only and might not reflect the current revision of the SuperNIC.
SuperNIC with Heatsink | Partner-Cooled SuperNIC |
| 
|
Item | Interface | Description |
ConnectX-8 Integrated Circuit | ||
2 | PCIe Gen 5.0 through x16 edge connector | |
3 | Network traffic is transmitted through the SuperNIC OSFP connectors. The networking connectors allow for the use of modules, optical and passive cable interconnect solutions. | |
4 | Two I/O LEDs per port to indicate speed and link status | |
FRU EEPROM capacity 16KB | ||
Allows BMC connectivity using MCTP over SMBus or MCTP over PCIe protocols. | ||
Voltage supply pins that feed onboard regulators. | ||
Controls the networking port logic LED (LED 1) and implements the OCP 3.0 host scan chain. | ||
Dissipates the heat. |
ConnectX-8 IC
The ConnectX-8 family of IC devices delivers InfiniBand and Ethernet connectivity paired with best-in-class hardware capabilities that accelerate and secure cloud and data-center workloads.
NVIDIA Multi-HostTM Support
In addition to building exceptionally high bandwidth to the data center, the ConnectX-8 device enables leveraging this speed across the entire data center utilizing its NVIDIA Multi-Host feature.
Using its 16-lane PCI Express interface, a single ConnectX-8 device can provide 400GbE interconnect for up to four independent hosts without any performance degradation.
The figure below shows a ConnectX-8 device with NVIDIA Multi-Host connected to four separate hosts, each with a PCIe x4 interface, on one side to a switch on the other side.
The below bifurcation are optional for the adapter's x16 PCIe interface:
x1 PCIe x16, x1 PCIe x8, x1 PCIe x4
x2 PCIe x8, x2 PCIe x4, x2 PCIe x2, x2 PCIe x1
x4 PCIe x4, x4 PCIe x2, x4 PCIe x1
Multi-host capable cards also support Socket-Direct applications and work as regular Single-Host cards, depending on the type of server they are plugged into, assuming the server complies with the OCP 3.0 spec.
According to the OCP 3.0 spec, the SuperNIC advertises its capability through the PRSNTB[3:0]# pins. The server determines the configuration through the BIF[2:0]# pins, which it drives to the SuperNIC.
The NVIDIA OCP3.0 card has an internal logic that uses the BIF[2:0]# data and determines the correct operating mode to boot at. The combination of the PRSNTB[3:0]# and BIF[2:0]# pins deterministically sets the PCIe lane width for a given combination of OCP 3.0 cards and baseboard. The logic and the decoding table can be found in the OCP 3.0 spec (Chapter 3.5 PCIe Bifurcation Mechanism).
For example:
the NVIDIA OCP 3.0 Multi-host adapter drives 0100 on PRSNTB[3:0]# to the server.
If Server Drivers | Adapter PCIe Mode |
000 | Single-Host Mode: x1 PCIe x16 |
001 | Socket Direct Mode: x2 PCIe x8 |
010 | Socket Direct Mode: x4 PCIe x4 |
101 | Multi-Host Mode: x2 PCIe x8 |
110 | Multi-Host Mode: x4 PCIe x4 |
PCI Express Interface
ConnectX-8 SuperNIC supports PCI Express Gen6 (5.0 and 4.0 compatible) through an x16 edge connector. The following lists host interface features:
PCIe Gen6 or Gen5 (up to x16 PCIe lanes)
NVIDIA Multi-Host™ (up to 4 hosts)
PCIe switch downstream port containment (DPC)
MSI/MSI-X
Networking Interfaces
The SuperNIC includes special circuits to protect the SuperNIC/server from ESD shocks when plugging copper cables.
Ethernet and InfiniBand traffic is transmitted through the networking connectors (OSFP) on the SuperNIC.
Protocol | Specifications |
Ethernet | The network ports comply with the IEEE 802.3 Ethernet standards in Features and Benefits. |
InfiniBand | The network ports are compliant with the InfiniBand Architecture Specification, Release 1.7. |
Networking Ports LEDs Specifications
There are two I/O LEDs per port to indicate port speed and link status.
LED 1 is a bi-color LED (Yellow and Green)
LED 2 is a single-color LED (Green)
Single-Port OSFP Thumbscrew Bracket | Single-Port OSFP Internal Lock Bracket |
| 
|
State | Bi-Color LED (Yellow/Green) | Single Color LED (Green) | |||||||||
Beacon command for locating the SuperNIC | 1Hz blinking Yellow | OFF | |||||||||
Error | 4Hz blinking Yellow Indicates an error with the link. The error can be one of the following:
| ON | |||||||||
Link Up | In full port speed: the Green LED is solid In less than full port speed: the Yellow LED is solid | ON | |||||||||
Physical Activity | The Green LED will blink. | Blinking |
FRU EEPROM
FRU EEPROM allows the baseboard to identify different types of OCP 3.0 cards. FRU EEPROM is accessible through SMCLK and SMDATA. FRU EEPROM address is defined according to SLOT_ID0 and SLOT_ID1 and its capacity is 16KB.
SMBus Interface
ConnectX-8 technology maintains support for manageability through a BMC. ConnectX-8 OCP 3.0 adapter can be connected to a BMC using MCTP over SMBus or MCTP over PCIe protocols as if it is a standard NVIDIA OCP 3.0 adapter. For configuring the adapter for the specific manageability solution in use by the server, please contact NVIDIA Support.
Voltage Regulators
The SuperNIC incorporates a CPLD device that implements the OCP 3.0 host scan chain and c ontrols the networking port logic LED (LED0) . It draws its power supply from the 3.3V_EDGE and 12V_EDGE rails.