Interfaces

NVIDIA ConnectX-6 InfiniBand Ethernet Adapter Cards User Manual

The network ports of the ConnectX®-6 adapter cards are compliant with the InfiniBand Architecture Specification, Release 1.3. InfiniBand traffic is transmitted through the cards' QSFP56 connectors.

Warning

The adapter card includes special circuits to protect from ESD shocks to the card/server when plugging copper cables.

The network ports of the ConnectX-6 adapter card are compliant with the IEEE 802.3 Ethernet standards listed in Features and Benefits. Ethernet traffic is transmitted through the QSFP56/QSFP connectors on the adapter card.

ConnectX®-6 adapter cards support PCI Express Gen 3.0/4.0 (1.1 and 2.0 compatible) through x8/x16 edge connectors. The device can be either a master initiating the PCI Express bus operations or a subordinate responding to PCI bus operations.
The following lists PCIe interface features:

  • PCIe Gen 3.0 and 4.0 compliant, 2.0 and 1.1 compatible

  • 2.5, 5.0, 8.0, or 16.0 GT/s link rate x16/x32

  • Auto-negotiates to x32, x16, x8, x4, x2, or x1

  • Support for MSI/MSI-X mechanisms

Warning

The adapter card includes special circuits to protect from ESD shocks to the card/server when plugging copper cables.

There are two I/O LEDs per port:

  • LED 1 and 2: Bi-color I/O LED which indicates link status. LED behavior is described below for Ethernet and InfiniBand port configurations.

  • LED 3 and 4: Reserved for future use.

    image2019-7-1_11-27-30.png

LED1 and LED2 Link Status Indications - Ethernet Protocol:

LED Color and State

Description

Off

A link has not been established

Beacon command for locating the adapter card

1Hz blinking Yellow

Error

4Hz blinking Yellow Indicates an error with the link. The error can be one of the following:

Error Type

Description

LED Behavior

I2C

I2C access to the networking ports fails

Blinks until error is fixed

Over-current

Over-current condition of the networking ports

Blinks until error is fixed

Solid green

Indicates a valid link with no active traffic

Blinking green

Indicates a valid link with active traffic

LED1 and LED2 Link Status Indications - InfiniBand Protocol:

LED Color and State

Description

Off

A link has not been established

Beacon command for locating the adapter card

1Hz blinking Yellow

Error

4Hz blinking Yellow Indicates an error with the link. The error can be one of the following:

Error Type

Description

LED Behavior

I2C

I2C access to the networking ports fails

Blinks until error is fixed

Over-current

Over-current condition of the networking ports

Blinks until error is fixed

Solid amber

Indicates an active link

Solid green

Indicates a valid (data activity) link with no active traffic

Blinking green

Indicates a valid link with active traffic

The heatsink is attached to the ConnectX-6 IC to dissipate the heat from the ConnectX-6 IC. It is attached either by using four spring-loaded push pins that insert into four mounting holes or by screws.
ConnectX-6 IC has a thermal shutdown safety mechanism that automatically shuts down the ConnectX-6 card in cases of high-temperature events, improper thermal coupling or heatsink removal.
For the required airflow (LFM) per OPN, please refer to Specifications.

For MCX653105A-HDAL and MCX653106A-HDAL cards, the heatsink is compatible with a cold plate for liquid-cooled Intel® Server System D50TNP platforms only.

ConnectX-6 technology maintains support for manageability through a BMC. ConnectX-6 PCIe stand-up adapter can be connected to a BMC using MCTP over SMBus protocol as if it is a standard NVIDIA PCIe stand-up adapter. For configuring the adapter for the specific manageability solution in use by the server, please contact NVIDIA Support.

The voltage regulator power is derived from the PCI Express edge connector 12V supply pins. These voltage supply pins feed on-board regulators that provide the necessary power to the various components on the card.

© Copyright 2023, NVIDIA. Last updated on Nov 9, 2023.