NVIDIA External Multi-Host Adapter Kit for OCP 3.0 User Manual
NVIDIA External Multi-Host Adapter Kit for OCP 3.0 User Manual

Introduction

This is the User Guide for NVIDIA External Multi-Host Adapter Kit for OCP 3.0 with a connectivity of up to 4 standard servers.

Important

Caution: Powering up the MiniSAS auxiliary cards before the OCP 3.0 multi-host board is powered up may cause not power-up the card.

NVIDIA Multi-Host technology enables connecting up to 4 compute / storage hosts to a single OCP 3.0 multi-host adapter. The deployment of multi-host platforms significantly reduces the overall number of data-center network connections, enabling great infrastructure efficiency and simplicity with CAPEX and OPEX cost savings. NVIDIA Multi-Host technology is built into ConnectX SmartNICs and BlueField DPUs and is ideal for high-performance, compute-intensive, data-center environments delivering cloud, web 2.0, and telecom services.

The externally connected multi-host solution leverages the same multi-host technology built into the network adapter ASIC. The external Mini-SAS harnesses make up the connectivity between each host and the network card. The solution is perfectly positioned for highly dense 4-node 2U chassis systems, enabling 25Gb/s connectivity per connected node, as illustrated in the figure below.

The externally connected Multi-host solution offers a superior price/performance ratio for both new and existing scale-out computing fabrics, while dramatically reducing the total cost of ownership on the following:

  • Network adapters, operating as a single network adapter, serve up to 4 nodes.

  • Network switch ports, operating as a single switch port, serve up to 4 nodes.

  • Active cabling, as a single cable now serves up to four nodes.

  • Rack space, power and cooling, attributed to the overall reduction in network connections in the data center.

Warning

The below figure is for illustration purposes only. The OCP 3.0 card is not included in the package.

image2020-12-8_10-44-53.png

Power Supply

Minimum of 120W external ATX power supply source (not included in the package contents)

External Fan

Critical requirement: at least one external fan (not included in the package contents).
Please refer to the airflow requirements listed in the specifications table.

Operating System

OpenFabrics Enterprise Distribution (OFED)

Connectivity

  • Interoperable with up to 100Gb/s Ethernet switches

  • Passive copper cable with ESD protection

  • Powered connectors for optical and active cable support

Prior to unpacking your product, it is important to make sure your system meets all the requirements listed above for a smooth installation. Be sure to inspect each piece of equipment shipped in the packing box. If anything is missing or damaged, contact your reseller.

Item

Illustration

Cards

1x Multi-Host OCP 3.0 adapter

image2020-12-8_10-47-35.png

4x MiniSAS Auxilliary boards

image2020-12-8_10-49-14.png

Cables

x4 MiniSAS HD cables

shopping.png

USB type-A to Mini-USB type B cable

image2020-12-1_8-38-38.png

Accessories

I/O Panel(a)

image2020-12-1_8-38-43.png

(a) The I/O panel is included in the package and can be used when inserting the product into a server. Currently, the option is not supported.

Model

OCP 3.0 Multi-Host Kit

Part Number

MEMH-OCP3-AK70

Network Connector Type

Supports any OCP 3.0 adapter card

Dimensions

Multi-Host Board: 6.69in x 6.69in (170mm x 170mm)
OCP 3.0 Adapter Card
: Small Form Factor (SFF) OCP 3.0 - 4.52 in. x 2.99 in (115.00mm x 76.00mm)
4x MiniSAS Auxiliary Cards: 1.96 in. x 3.54 in (50.0mm x 90.0 mm)
4x Mini SAS Harnesses: 70cm each


For more detailed information see Specifications.

Warning

This section describes hardware features and capabilities. Please refer to the relevant driver and/or firmware release notes for feature availability.

Warning

For visualization scenarios, please contact NVIDIA support.

Feature

Description

PCI Express (PCIe)

Uses the following PCIe interfaces:

  • OCP 3.0 Card: PCIe Gen 3.0 SERDES @ 8.0 GT/s through x8 edge connector

  • MiniSAS Auxilliary Cards: 4x PCIe Gen 3.0 SERDES @ 8.0GT/s through x4 edge connector

Memory

  • SPI Quad - includes 128Mbit SPI Quad Flash device (W25Q128JVSIQ device WINBOND-NUVOTON).

  • FRU EEPROM - Provides the parameters and personality of the card. The EEPROM capacity is 128Kbit. FRU I2C address is (0x50) and is accessible through the PCIe SMBus.

NC-SI over RMII

A Network Controller Sideband Interface (NC-SI) is a combination of logical and physical paths that interconnect the Management Controller and Network Controller(s) for the purpose of transferring management communication traffic among them. NC-SI includes commands and associated responses which the Management Controller uses to control the status and operation of the Network Controller(s). NC-SI also includes a mechanism for transporting management traffic and asynchronous notifications. Please connect an ethernet cable from the management console to J104 of the OCP 3.0 host adapter.

Wake-on-LAN

Wake-on-LAN (WoL) is a feature that allows a network professional to remotely power on a server (4 servers together) or be awakenedby a network message. The feature is applicable only when the OCP 3.0 card supports the feature.

Overlay Networks

In order to better scale their networks, data center operators often create overlay networks that carry traffic from individual virtual machines over logical tunnels in encapsulated formats such as NVGRE and VXLAN. While this solves network scalability issues, it hides the TCP packet from the hardware offloading engines, placing higher loads on the host CPU. OCP Multi-Host card effectively addresses this by providing advanced NVGRE and VXLAN hardware offloading engines that encapsulate and de-capsulate the overlay protocol.

RDMA and RDMA over Converged Ethernet (RoCE)

OCP Multi-Host card, utilizing IBTA RDMA (Remote Data Memory Access) and RoCE (RDMA over Converged Ethernet) technology, delivers low-latency and high performance over Band and Ethernet networks. Leveraging data center bridging (DCB) capabilities, as well as OCP Multi-Host card, advanced congestion control hardware mechanisms, RoCE provides efficient low-latency RDMA services over Layer 2 and Layer 3 networks.

NVIDIA PeerDirect™

PeerDirect™ communication provides high-efficiency RDMA access by eliminating unnecessary internal data copies between components on the PCIe bus (for example, from GPU to CPU), and therefore significantly reduces application run time. OCP Multi-Host card advanced acceleration technology enables higher cluster efficiency and scalability to tens of thousands of nodes.

CPU Offload

Adapter functionality enabling reduced CPU overhead allowing more available CPU for computation tasks.

Open VSwitch (OVS) offload using ASAP2

• Flexible match-action flow tables
• Tunneling encapsulation/decapsulation

Quality of Service (QoS)

Support for port-based Quality of Service enabling various application requirements for latency and SLA.

Storage Acceleration

A consolidated compute and storage network achieves significant cost-performance advantages over multi-fabric networks. Standard block and file access protocols can leverage InfiniBand RDMA for high-performance storage access.
• NVMe over Fabric offloads for the target machine
• Erasure Coding
• T10-DIF Signature Handover

SR-IOV

OCP Multi-Host card SR-IOV technology provides dedicated adapter resources and guaranteed isolation and protection for virtual machines (VM) within the server.

High-Performance Accelerations

• Tag Matching and Rendezvous Offloads
• Adaptive Routing on Reliable Transport
• Burst Buffer Offloads for Background Checkpointing

© Copyright 2023, NVIDIA. Last updated on May 22, 2023.