image image image image image

On This Page

This is the User Guide for Ethernet adapter cards based on the ConnectX®-5 VPI integrated circuit device for OCP Spec 2.0. These adapters connectivity provide the highest performing low latency and most flexible interconnect solution for PCI Express Gen 3.0/4.0 servers used in Enterprise Data Centers and High-Performance Computing environments.

The following provides the ordering part number, port speed, number of ports, and PCI Express speed. 

Products Overview

ModelConnectX-5 VPI Cards for OCP Spec 2.0
Part NumberMCX545A-ECANMCX545B-ECANMCX545M-ECANMCX546A-EDAN
OCP Spec 2.0 TypeType 2 Stacking Height Type 1 Stacking HeightType 2 Stacking HeightType 2 Stacking Height

Single/Multi-Host

Single-HostMulti-HostSingle-Host
Host Management
Network Connector Type

Single-port QSFP28

Dual-port belly-to-belly QSFP28

Data Rate

InfiniBand/VPI: SDR/DDR/QDR/FDR/EDR

Ethernet:10/25/40/50/100 Gb/s
PCI Express Connectors

PCIe Gen 3.0 x16
SERDES @ 8.0GT/s

PCIe Gen 3.0 / 4.0 x16
SERDES @ 8.0GT/s / 16.0GT/s

Dimensions

PCIe Half Height, Half Length
3.07 in. x 4.33 in (78.00mm x 110.05mm)

RoHSRoHS Compliant
IC Part Number MT28808A0-FCCF-EV
Device ID

4119 for Physical Function (PF)
4120 for Virtual Function (VF)

4121 for Physical Function (PF)
4122 for Virtual Function (VF)

For more detailed information see Specifications.

OCP Spec 2.0 Stacking Heights

OCP Spec 2.0 Type 1 Stacking Height 

This section applies to MCX542A-ACAN.

The ingle-port 100Gb/s adapter card comply with OCP Spec 2.0 Type 1 with 8mm stacking height.

OCP Spec 2.0 Type 2 Stacking Height 

This figure applies to MCX545A-ECAN, MCX545M-ECAN.

The single-port 100Gb/s adapter cards follow OCP Spec 2.0 Type 2 with 12mm stacking height.

Single-port Card

Features and Benefits

This section describes hardware features and capabilities. Please refer to the relevant driver and/or firmware release notes for feature availability.

FeatureDescription
PCI Express (PCIe)Uses PCIe Gen 3.0 (8GT/s) or Gen 4.0 (16GT/s) through x16 lanes. Gen 1.1 and 2.0 compatible.

100Gb/s Virtual Protocol
Interconnect (VPI) Adapter

ConnectX-5 offers the highest throughput VPI adapter, supporting EDR 100Gb/s InfiniBand and 100Gb/s Ethernet and enabling any standard networking, clustering, or storage to operate seamlessly over any converged network leveraging a consolidated software stack.
InfiniBand EDRA standard InfiniBand data rate, where each lane of a 4X port runs a bit rate of 25.78125Gb/s with a 64b/66b encoding, resulting in an effective bandwidth of 100Gb/s.
Up to 100 Gigabit Ethernet

NVIDIA adapters comply with the following IEEE 802.3 standards:

• 100GbE/ 50GbE / 40GbE / 25GbE / 10GbE / 1GbE
• IEEE 802.3bj, 802.3bm 100 Gigabit Ethernet
• IEEE 802.3by, Ethernet Consortium25, 50 Gigabit Ethernet, supporting all FEC modes
• IEEE 802.3ba 40 Gigabit Ethernet
• IEEE 802.3by 25 Gigabit Ethernet
• IEEE 802.3ae 10 Gigabit Ethernet
• IEEE 802.3ap based auto-negotiation and KR startup
• Proprietary Ethernet protocols (20/40GBASE-R2, 50GBASE-R4)
• IEEE 802.3ad, 802.1AX Link Aggregation
• IEEE 802.1Q, 802.1P VLAN tags and priority
• IEEE 802.1Qau (QCN)
• Congestion Notification
• IEEE 802.1Qaz (ETS)
• IEEE 802.1Qbb (PFC)
• IEEE 802.1Qbg
• IEEE 1588v2
• Jumbo frame support (9.6KB)

Memory
  • SPI - includes 128Mb SPI Flash device (W25Q128FVSIG by WINBOND-NUVOTON).
  • FRU EEPROM capacity is 2Kb. FRU I2C address is (0x50) and is accessible through the PCIe SMBus. Note: Address 0x58 is reserved.)
Overlay NetworksIn order to better scale their networks, data center operators often create overlay networks that carry traffic from individual virtual machines over logical tunnels in encapsulated formats such as NVGRE and VXLAN. While this solves network scalability issues, it hides the TCP packet from the hardware offloading engines, placing higher loads on the host CPU. ConnectX-5 effectively addresses this by providing advanced NVGRE and VXLAN hardware offloading engines that encapsulate and de-capsulate the overlay protocol.
RDMA and RDMA over Converged Ethernet (RoCE)ConnectX-5, utilizing IBTA RDMA (Remote Data Memory Access) and RoCE (RDMA over Converged Ethernet) technology, delivers low-latency and high-performance over Band and Ethernet networks. Leveraging data center bridging (DCB) capabilities as well as ConnectX-5 advanced congestion control hardware mechanisms, RoCE provides efficient low-latency RDMA services over Layer 2 and Layer 3 networks.
NVIDIA PeerDirect™PeerDirect™ communication provides high-efficiency RDMA access by eliminating unnecessary internal data copies between components on the PCIe bus (for example, from GPU to CPU), and therefore significantly reduces application run time. ConnectX-5 advanced acceleration technology enables higher cluster efficiency and scalability to tens of thousands of nodes.
CPU OffloadAdapter functionality enabling reduced CPU overhead allowing more available CPU for computation tasks.
Open vSwitch (OVS) offload using ASAP2(TM)
• Flexible match-action flow tables
• Tunneling encapsulation/decapsulation
Quality of Service (QoS)Support for port-based Quality of Service enabling various application requirements for latency and SLA.
Hardware-based I/O VirtualizationConnectX-5 provides dedicated adapter resources and guaranteed isolation and protection for virtual machines within the server.
Storage AccelerationA consolidated compute and storage network achieves significant cost-performance advantages over multi-fabric networks. Standard block and file access protocols can leverage RDMA for high-performance storage access.
• NVMe over Fabric offloads for the target machine
• Erasure Coding
• T10-DIF Signature Handover
SR-IOVConnectX-5 SR-IOV technology provides dedicated adapter resources and guaranteed isolation and protection for virtual machines (VM) within the server.
NC-SIThe adapter supports a Network Controller Sideband Interface (NC-SI), MCTP over SMBus and MCTP over PCIe - Baseboard Management Controller interface.
High-Performance Accelerations• Tag Matching and Rendezvous Offloads
• Adaptive Routing on Reliable Transport
• Burst Buffer Offloads for Background Checkpointing
Wake-on-LAN (WoL)Supported
Reset-on-LAN (RoL)Supported


Multi-Host Technology 

Supported in MCX545M-ECAN.

ConnectX®-5 adapter card specifically designed for supported servers implements Multi-Host technology to deliver direct and independent PCIe connections to each of the four CPUs in the server.

The ConnectX-5 PCIe x16 interface is separated into four independent PCIe x4 interfaces. Each interface is connected to a separate host with no performance degradation.
Connecting server CPUs directly to the network delivers performance gain as each CPU can send and receive network traffic independently without the need to send network data to other CPUs using QPI bus.

Operating Systems/Distributions

In MCX545M-ECAN, only OFED is supported.

  • RHEL/CentOS
  • Windows
  • FreeBSD
  • VMware
  • OpenFabrics Enterprise Distribution (OFED)
  • OpenFabrics Windows Distribution (WinOF-2)

Connectivity

  • Interoperable with 1/10/25/40/50/100 Gb/s Ethernet switches
  • Passive copper cable with ESD protection
  • Powered connectors for optical and active cable support