NVIDIA ConnectX-4 Lx Ethernet Adapter Cards User Manual
NVIDIA ConnectX-4 Lx Ethernet Adapter Cards User Manual

Introduction

This is the User Guide for Ethernet adapter cards based on the ConnectX®-4 Lx integrated circuit device. These adapters connectivity provide the highest performing low latency and most flexible interconnect solution for PCI Express Gen 3.0 servers used in Enterprise Data Centers and High-Performance Computing environments.

The following provides the ordering part number, port speed, number of ports, and PCI Express speed. Each adapter is shipped with two bracket heights - short and tall (assembled).

ConnectX-4 Lx 10Gb/s Ethernet Adapter Cards

Model

ConnectX-4 Lx 10Gb/s Ethernet Adapter Cards

Part Number

MCX4111A-XCAT

MCX4121A-XCAT

MCX4121A-XCHT

Dimensions

2.71 in. x 5.6 in. (68.90mm x 142.24 mm) – Low profile

PCI Express Connectors

PCIe Gen 3.0 SERDES @ 8.0GT/s x8 lanes

Ethernet Data Rate

1/10 Gb/s

1/10 Gb/s

1/10 Gb/s

Network Connector Type

Single-port SFP28

Dual-port SFP28

Dual-port SFP28

UEFI

-

-

Host Management

-

-

RoHS

RoHS Compliant

IC Part Number

MT27712A0-FDCF-XE

Device ID

4117 for Physical Function (PF)

4118 for Virtual Function (VF)

ConnectX-4 Lx 25Gb/s Ethernet Adapter Cards

Model

ConnectX-4 Lx 25Gb/s Ethernet Adapter Cards

Part Number

MCX4111A-ACAT

MCX4111A-ACUT

MCX4121A-ACAT

MCX4121A-ACUT

MCX4121A-ACST

MCX4121A-ACHT

Network Connector Type

1Port SFP28

1Port SFP28

2Ports SFP28

2Ports SFP28

2Ports SFP28

2Ports SFP28

Ethernet Data Rate

1/10/25 Gb/s

PCI Express Connectors

PCIe Gen 3.0 SERDES @ 8.0GT/s x8 Lanes

Dimensions

2.71 in. x 5.6 in. (68.90mm x 142.24 mm) – low profile

UEFI

-

-

-

Secure Firmware

-

-

-

-

-

Host Management

-

-

-

-

-

RoHS

RoHS Compliant

IC Part Number

MT27712A0-FDCF-AE

Device ID

4117 for Physical Function (PF)

4118 for Virtual Function (VF)

ConnectX-4 Lx 40/50 Gb/s Ethernet Adapter Cards

Model

ConnectX-4 Lx 40/50 Gb/s Ethernet Adapter Cards

Part Number

MCX4131A-BCAT

MCX4131A-GCAT

Ethernet Data Rate

1/10/25/40 Gb/s

1/10/25/40/50 Gb/s

Network Connector Type

Single-port QSFP28

PCI Express Connectors

PCIe Gen 3.0 x8 SerDes @ 8.0GT/s x8 Lanes

Dimensions

2.71 in. x 5.6 in. (68.90mm x 142.24 mm) – low profile

RoHS

RoHS Compliant

IC Part Number

MT27711A0-FDCF-BE

MT27711A0-FDCF-GE

Device ID

4117 for Physical Function (PF)

4118 for Virtual Function (VF)

For more detailed information see Specifications.

Warning

This section describes hardware features and capabilities. Please refer to the relevant driver and/or firmware release notes for feature availability.

Feature

Description

PCI Express (PCIe)

Uses PCIe Gen 3.0 (8GT/s) through an x8 edge connector. Gen 1.1 and 2.0 compatible.

Up to 50 Gigabit Ethernet

NVIDIA adapters comply with the following IEEE 802.3 standards:

• 50GbE / 40GbE / 25GbE / 10GbE / 1GbE

• IEEE 802.3by, Ethernet Consortium25, 50 Gigabit Ethernet, supporting all FEC modes

• IEEE 802.3ba 40 Gigabit Ethernet

• IEEE 802.3by 25 Gigabit Ethernet

• IEEE 802.3ae 10 Gigabit Ethernet

• IEEE 802.3ap based auto-negotiation and KR startup

• Proprietary Ethernet protocols (20/40GBASE-R2, 50GBASE-R4)

• IEEE 802.3ad, 802.1AX Link Aggregation

• IEEE 802.1Q, 802.1P VLAN tags and priority

• IEEE 802.1Qau (QCN)

• Congestion Notification

• IEEE 802.1Qaz (ETS)

• IEEE 802.1Qbb (PFC)

• IEEE 802.1Qbg

• IEEE 1588v2

• Jumbo frame support (9.6KB)

Memory

  • PCI Express - stores and accesses Ethernet fabric connection information and packet data.

  • SPI - includes one 16MB SPI Flash device (W25Q128FVSIG device by WINBOND-NUVOTON).

Overlay Networks

In order to better scale their networks, data center operators often create overlay networks that carry traffic from individual virtual machines over logical tunnels in encapsulated formats such as NVGRE and VXLAN. While this solves network scalability issues, it hides the TCP packet from the hardware offloading engines, placing higher loads on the host CPU. ConnectX-4 Lx effectively addresses this by providing advanced NVGRE and VXLAN hardware offloading engines that encapsulate and de-capsulate the overlay protocol.

RDMA and RDMA over Converged Ethernet (RoCE)

ConnectX-4 Lx, utilizing IBTA RDMA (Remote Data Memory Access) and RoCE (RDMA over Converged Ethernet) technology, delivers low-latency and high-performance over Band and Ethernet networks. Leveraging data center bridging (DCB) capabilities, as well as ConnectX-4 Lx, advanced congestion control hardware mechanisms, RoCE provides efficient low-latency RDMA services over Layer 2 and Layer 3 networks.

NVIDIA PeerDirect™

PeerDirect™ communication provides high-efficiency RDMA access by eliminating unnecessary internal data copies between components on the PCIe bus (for example, from GPU to CPU), and therefore significantly reduces application run time. ConnectX-4 Lx advanced acceleration technology enables higher cluster efficiency and scalability to tens of thousands of nodes.

CPU Offload

Adapter functionality enabling reduced CPU overhead allowing more available CPU for computation tasks.

Quality of Service (QoS)

Support for port-based Quality of Service enabling various application requirements for latency and SLA.

Hardware-based I/O Virtualization

ConnectX-4 Lx provides dedicated adapter resources and guaranteed isolation and protection for virtual machines within the server.

Storage Acceleration

A consolidated compute and storage network achieves significant cost-performance advantages over multi-fabric networks. Standard block and file access protocols can leverage RDMA for high-performance storage access.

• NVMe over Fabric offloads for the target machine

SR-IOV

ConnectX-4 Lx SR-IOV technology provides dedicated adapter resources and guaranteed isolation and protection for virtual machines (VM) within the server.

NC-SI

The adapter supports a Network Controller Sideband Interface (NC-SI), MCTP over SMBus and MCTP over PCIe - Baseboard Management Controller interface.

High-Performance Accelerations

• Tag Matching and Rendezvous Offloads

• Adaptive Routing on Reliable Transport

• Burst Buffer Offloads for Background Checkpointing

UEFI

UEFI is a standard firmware interface designed to replace BIOS. NVIDIA UEFI Network driver allows boot over the network via PXE (Preboot eXecution Environment). This network driver allows remote boot over Ethernet, or Boot over iSCSI (Bo-iSCSI) in UEFI mode, and also supports the Secure Boot standard. The UEFI Network driver allows IT managers the flexibility to deploy servers with a single adapter card into Ethernet networks while also enabling booting from LAN or remote storage targets. In addition to boot capabilities, NVIDIA UEFI Network driver provides firmware management and diagnostic protocols compliant with the UEFI specification.

For further information, refer to the NVIDIA PreBoot Drivers User Manual.

Supported in MCX4111A-ACUT, MCX4121A-XCHT, MCX4121A-ACUT and MCX4121A-ACHT.

Secure Firmware

A “Secure firmware update” is the ability of a device to verify digital signatures of new firmware binaries, in order to assure that only officially approved versions can be installed from the host, the network or a Board Management Controller (BMC). The firmware of devices with “secure firmware update” functionality (secure FW), restricts access to specific commands and registers that can be used to modify the firmware binary image on the flash, as well as commands that can jeopardize security in general.

For further information, refer to the MFT User Manual.

Supported in MCX4121A-ACST.

  • RHEL/CentOS

  • Windows

  • FreeBSD

  • VMware

  • OpenFabrics Enterprise Distribution (OFED)

  • OpenFabrics Windows Distribution (WinOF-2)

  • Interoperable with 1/10/25/40/50 Gb/s Ethernet switches

  • Passive copper cable with ESD protection

  • Powered connectors for optical and active cable support

© Copyright 2023, NVIDIA. Last updated on Jan 14, 2024.