This is the User Guide for Ethernet adapter cards based on the ConnectX®-6 integrated circuit device for OCP Spec 3.0. These adapters connectivity provide the highest performing low latency and most flexible interconnect solution for PCI Express Gen 3.0/4.0 servers used in Enterprise Data Centers and High-Performance Computing environments.
In some of the OCP 3.0 cards, the insertion force that is required to install the card into a particular PCI connector may exceed the maximum insertion force that is allowed by the connector’s spec. NVIDIA is updating the OCP 3.0 cards to ensure the proper insertion process.
The following table provides the ordering part number, port speed, number of ports, and PCI Express speed.
Please note the following OPN includes an internal lock bracket. For other retention mechanisms brackets, please contact NVIDIA.
|Model||ConnectX-6 EN Card for OCP Spec 3.0|
|Network Connector Type|
|Ethernet Data Rate||10/25/40/50/100/200 Gb/s|
|PCI Express Connectors|
PCIe Gen 3.0 / 4.0 x16
Small Form Factor (SFF) OCP 3.0
|Retention Mechanism||Internal Lock|
4123 for Physical Function (PF)
For more detailed information see Specifications.
Features and Benefits
This section describes hardware features and capabilities. Please refer to the relevant driver and/or firmware release notes for feature availability.
|PCI Express (PCIe)||Uses PCIe Gen 3.0 (8GT/s) or Gen 4.0 (16GT/s) through x16 edge connector.|
|Up to 200 Gigabit Ethernet|
NVIDIA adapters comply with the following IEEE 802.3 standards:
• 200GbE / 100GbE / 50GbE / 40GbE / 25GbE / 10GbE / 1GbE
|Overlay Networks||In order to better scale their networks, data center operators often create overlay networks that carry traffic from individual virtual machines over logical tunnels in encapsulated formats such as NVGRE and VXLAN. While this solves network scalability issues, it hides the TCP packet from the hardware offloading engines, placing higher loads on the host CPU. ConnectX-6 effectively addresses this by providing advanced NVGRE and VXLAN hardware offloading engines that encapsulate and de-capsulate the overlay protocol.|
|RDMA and RDMA over Converged Ethernet (RoCE)||ConnectX-6, utilizing IBTA RDMA (Remote Data Memory Access) and RoCE (RDMA over Converged Ethernet) technology, delivers low-latency and high-performance over Band and Ethernet networks. Leveraging data center bridging (DCB) capabilities as well as ConnectX-6 advanced congestion control hardware mechanisms, RoCE provides efficient low-latency RDMA services over Layer 2 and Layer 3 networks.|
|NVIDIA PeerDirect®||PeerDirect® communication provides high-efficiency RDMA access by eliminating unnecessary internal data copies between components on the PCIe bus (for example, from GPU to CPU), and therefore significantly reduces application run time. ConnectX-6 advanced acceleration technology enables higher cluster efficiency and scalability to tens of thousands of nodes.|
Adapter functionality enables reduced CPU overhead leaving more CPU resources available for computation tasks.
|Quality of Service (QoS)||Support for port-based Quality of Service enabling various application requirements for latency and SLA.|
|Hardware-based I/O Virtualization||ConnectX-6 provides dedicated adapter resources and guaranteed isolation and protection for virtual machines within the server.|
|Storage Acceleration||A consolidated compute and storage network achieves significant cost-performance advantages over multi-fabric networks. Standard block and file access protocols can leverage RDMA for high-performance storage access.|
• NVMe over Fabric offloads for the target machine
• Erasure Coding
• T10-DIF Signature Handover
|SR-IOV||ConnectX-6 SR-IOV technology provides dedicated adapter resources and guaranteed isolation and protection for virtual machines (VM) within the server.|
|NC-SI||The adapter supports a Network Controller Sideband Interface (NC-SI), MCTP over SMBus and MCTP over PCIe - Baseboard Management Controller interface.|
|High-Performance Accelerations||• Tag Matching and Rendezvous Offloads|
• Adaptive Routing on Reliable Transport
• Burst Buffer Offloads for Background Checkpointing
The adapter supported Wake-on-LAN (WoL), a computer networking standard that allows an adapter to be turned on or awakened by a network message.
- OpenFabrics Enterprise Distribution (OFED)
- OpenFabrics Windows Distribution (WinOF-2)
- Interoperable with 1/10/25/40/50/100/200 Gb/s Ethernet switches
- Passive copper cable with ESD protection
- Powered connectors for optical and active cable support