PCI Express Pinouts Description for Single-Slot Socket Direct Card

Warning

This section applies to ConnectX-6 single-slot cards (MCX653105A-EFAT and MCX653106A-EFAT).

ConnectX-6 single-slot Socket Direct cards offer improved performance to dual-socket servers by enabling direct access from each CPU in a dual-socket server to the network through its dedicated PCIe interface. The PCIe x16 interface is split into two PCIe x8 in a row, such that each of the PCIe x8 lanes can be connected to a dedicated CPU in a dual-socket server. In such a configuration, Socket Direct brings lower latency and lower CPU utilization as the direct connection from each CPU to the network means the Interconnect can bypass a QPI (UPI) and the other CPU, optimizing performance and improving latency. CPU utilization is improved as each CPU handles only its own traffic and not traffic from the other CPU.

In order to allow this capability, a system with a special PCI Express x16 slot is required. Table 31 provides the pin definitions of the required four special PCIe pins.

PCIe pin#

Server Connection for 2x PCIe x8 mode

Server Connection for 1x PCIe x16 mode

Comments

B82

1Kohm pull up to 3.3VAUX

Either leave unconnected or pulled to GND

Configure the card to work at 2x PCIe x8 or 1x PCIe x16 modes

A32

P signal of differential PCIe clock (100MHz nominally) of the CPU which connects to PCIe lanes 15-8 of the PCIe connector

Not Connected

PCIe clock for ConnectX-6 PCIe bus lanes [15:8]

A33

N signal of differential PCIe clock (100MHz nominally) of the CPU which connects to PCIe lanes 15-8 of the PCIe connector

Not Connected

A50

PERST signal from the CPU which connects to PCIe lanes 15-8 of the PCIe connector

Not Connected

PERST (PCIe Reset) for ConnectX-6 PCIe bus lanes [15:8]

© Copyright 2023, NVIDIA. Last updated on Nov 9, 2023.