NVIDIA ConnectX-7 Adapter Cards User Manual
NVIDIA ConnectX-7 Adapter Cards User Manual

Supported Interfaces

This section describes the ConnectX-7 supported interfaces. Each numbered interface that is referenced in the figures is described in the following table with a link to detailed information.

Note

The below figures are for illustration purposes only and might not reflect the current revision of the adapter card.

Single-Port QSFP112 Adapter Cards

OPNs: MCX715105AS-WEAT

Dual-Port QSFP112 Adapter Cards

OPNs: MCX755106AS-HEAT, MCX755106AC-HEAT, MCX713106AC-CEAT,

MCX713106AS-CEAT, MCX713106AC-VEAT, MCX713106AS-VEAT

image2023-5-30_13-55-53-version-1-modificationdate-1706784194757-api-v2.png

image2022-2-10_15-37-35-version-1-modificationdate-1706784195433-api-v2.png

Single-Port OSFP Adapter Cards

OPNs: MCX75310AAS-NEAT, MCX75310AAC-NEAT, MCX75310AAS-HEAT,

MCX75510AAS-NEAT, MCX75510AAS-HEAT

Quad-Port SFP56 Cards

OPNs: MCX713104AC-ADAT, MCX713104AS-ADAT

image2022-9-21_18-55-8-version-1-modificationdate-1706784195097-api-v2.png

image2023-3-13_13-19-22-version-1-modificationdate-1706784197887-api-v2.png

Quad-port SFP56 Cards with PPS IN/OUT

OPN: MCX713114TC-GEAT

image2023-5-30_14-26-17-version-1-modificationdate-1706784194497-api-v2.png

Item

Interface

Description

1

ConnectX-7 IC 

ConnectX-7 Integrated Circuit

2

PCI Express Interface

PCIe Gen 4.0/5.0 through x16 edge connector

3

Networking Interfaces

Network traffic is transmitted through the adapter card networking connectors. The networking connectors allow for the use of modules, optical and passive cable interconnect solutions

4

Networking Ports LEDs 

Two I/O LEDs per port to indicate speed and link status

5

Cabline CA-II Plus Connectors 

In Socket-Direct ready cards, two Cabline CA-II plus connectors are populated to allow connectivity to an additional PCIe x16 Auxiliary card.

Applicable to OPNs: MCX715105AS-WEAT, MCX75510AAS-NEAT, MCX75510AAS-HEAT, MCX755106AS-HEAT and MCX755106AC-HEAT.

6

PPS IN/OUT Interface

Allows PPS IN/OUT

Applies to OPN: MCX713114TC-GEAT only.

ConnectX-7 IC

The ConnectX-7 family of adapter IC devices delivers InfiniBand and Ethernet connectivity paired with best-in-class hardware capabilities that accelerate and secure cloud and data-center workloads.

PCI Express Interface

ConnectX-7 adapter cards support PCI Express Gen 5.0 (4.0 and 3.0 compatible) through x16 edge connector. The following lists PCIe interface features:

  • PCIe Gen 5.0 compliant, 4.0, 3.0, 2.0 and 1.1 compatible

  • 2.5, 5.0, 8.0, 16.0 and 32GT/s link rate x16/x32 (Socket-Direct configuration)

  • Support for PCIe bifurcation: Auto-negotiates to x32, x16, x8, x4, x2, or x1

  • NVIDIA Multi-Host™ supports connection of up to 4x hosts

  • Transaction layer packet (TLP) processing hints (TPH)

  • PCIe switch Downstream Port Containment (DPC)

  • Advanced error reporting (AER)

  • Access Control Service (ACS) for peer-to-peer secure communication

  • Process Address Space ID (PASID)

  • Address translation services (ATS)

  • Support for MSI/MSI-X mechanisms

  • Support for SR-IOV

Networking Interfaces

Note

The adapter card includes special circuits to protect from ESD shocks to the card/server when plugging copper cables.

Protocol

Specifications

Ethernet

The network ports comply with the IEEE 802.3 Ethernet standards listed in Features and Benefits. Ethernet traffic is transmitted through the networking connectors on the adapter card.

InfiniBand

The network ports are compliant with the InfiniBand Architecture Specification, Release 1.5. InfiniBand traffic is transmitted through the cards' networking connectors.


Networking Ports LEDs Specifications

For the networking ports LEDs description, follow the below table depending on the ConnectX-7 SKU you have purchased.

SKU

LEDs Scheme

900-9X7AO-0003-ST0

900-9X7AO-00C3-STZ

Scheme 1: One Bi-Color LED

All cards

Scheme 2: Two LEDs

Scheme 1: One Bi-Color LED

There is one bi-color (Yellow and Green) I/O LED per port to indicate port speed and link status.

State

Bi-Color LED (Yellow/Green)

Beacon command for locating the adapter card

1Hz blinking Yellow

Error

4Hz blinking Yellow Indicates an error with the link. The error can be one of the following:

Error Type

Description

LED Behavior

I2C

I2C access to the networking ports fails

Blinks until error is fixed

Over-current

Over-current condition of the networking ports

Blinks until error is fixed

Physical Activity

The Green LED will blink.

Link Up

The Green LED will be solid.

Physical Up (IB Only)

The Yellow LED will be solid.


Scheme 2: Two LEDs

There are two I/O LEDs per port to indicate port speed and link status.

  • LED1 is a bi-color LED (Yellow and Green)

  • LED2 is a single-color LED (Green)

    image2023-3-1_17-6-12-version-1-modificationdate-1706784196907-api-v2.png

    image2023-3-1_17-6-37-version-1-modificationdate-1706784197290-api-v2.png

State

Bi-Color LED (Yellow/Green)

Single Color LED (Green)

Beacon command for locating the adapter card

1Hz blinking Yellow

OFF

Error

4Hz blinking Yellow Indicates an error with the link. The error can be one of the following:

Error Type

Description

LED Behavior

I2C

I2C access to the networking ports fails

Blinks until error is fixed

Over-current

Over-current condition of the networking ports

Blinks until error is fixed

ON

Physical Activity

The Green LED will blink.

Blinking

Link Up

In full port speed: the Green LED is solid

In less than full port speed: the Yellow LED is solid

ON

Cabline CA-II Plus Connectors

Info

Socket-Direct is currently not supported.

Note

Applies to OPNs: MCX755106AC-HEAT, MCX755106AS-HEAT, MCX75510AAS-HEAT, MCX75510AAS-NEAT.

The Cabline CA-II connectors on the Socket-Direct ready cards enable connectivity to an additional Auxiliary PCIe x16 Connection card through the Cabline CA-II harnesses.

PPS IN/OUT Interface

Note

Applicable to MCX713114TC-GEAT only.

Pulse Per Second (PPS) is an out-of-band signal used in synchronized systems. 5T technology support PPS-in and PPS-out on selected devices.

Selected ConnectX-7 adapter cards incorporate an integrated Hardware Clock (PHC) that allows the adapter to achieve sub-20u Sec accuracy and also offers many timing-related functions such as time-triggered scheduling or time-based SND accelerations (time-based ASAP²). Furthermore, 5T technology enables the software application to transmit fronthaul (ORAN) at high bandwidth. The PTP part supports the subordinate clock, master clock, and boundary clock. The PTP solution allows you to run any PTP stack on your host.

With respect to testing and measurements, selected ConnectX-7 adapters allow you to use the PPS-out signal from the onboard MMCX RA connecter. The adapter also allows measuring PTP in scale with the PPS-In signal. The PTP HW clock on the Network adapter is sampled on each PPS-In signal, and the timestamp is sent to the SW.

After the DPU installation, use two standard SMA plug 50Ohm cables to connect to the SMA connectors on the board. The cables are not included in the package. See the below example:

sma-version-1-modificationdate-1706784193797-api-v2.png


Clock IN/OUT Interface

Note

Applicable to MCX713114TC-GEAT only.

After the adapter card installation, use two standard MMCX 50Ohm, right angled, plugs to connect to the MMCX connectors on the board. The cables are not included in the package. See the below example:

cloclinout-version-1-modificationdate-1711440339943-api-v2.png


SMBus Interface

ConnectX-7 technology maintains support for manageability through a BMC. ConnectX-7 PCIe stand-up adapter can be connected to a BMC using MCTP over SMBus or MCTP over PCIe protocols as if it is a standard NVIDIA PCIe stand-up adapter. For configuring the adapter for the specific manageability solution in use by the server, please contact NVIDIA Support.

Voltage Regulators

The voltage regulator power is derived from the PCI Express edge connector 12V supply pins. These voltage supply pins feed on-board regulators that provide the necessary power to the various components on the card.

© Copyright 2024, NVIDIA. Last updated on Nov 22, 2023.