Interfaces
This document is preliminary and subject to change.
The systems support the following interfaces:
Data interfaces - InfiniBand
10/100/1000 MbE RJ45 management interface(s)
USB
RS232 Console port
I²C interface
Reset button
Status and Port LEDs
In order to review the full configuration options matrix, refer to Management Interfaces, PSUs and Fans.
The data interfaces use QSFP28 and QSFP56 connectors. The full list of interfaces per system is provided in Speed and Switching Capabilities.
Each QSFP28 port can be connected with a QSFP28 cable or connector for 40/56/100Gb/s. Each QSFP56 port can be connected with QSFP56 cable or connector for 40/56/100/200Gb/s. The system offers High Power class 4 transceivers support in all ports. You may use the following NVIDIA transceivers, designed for InfiniBand EDR links on up to 10km and 40km of single mode fiber:
OPN | Description |
MMA1L10-CR | NVIDIA optical transceiver, 100GbE, 100Gb/s, QSFP28, LC-LC, 1310nm, LR4 up to 10km |
SPQ-CE-ER-CDFL-M | 40km 100G QSFP28 ER Optical Transceiver |
Speed
InfiniBand speed is auto-adjusted by the InfiniBand protocol. NVIDIA systems support QDR/FDR/EDR/HDR InfiniBand.
FDR is an InfiniBand data rate, where each lane of a 4X port runs a bit rate of 14.0625Gb/s with 64b/66b encoding, resulting in an effective bandwidth of 56.25Gb/s.
EDR is an InfiniBand data rate, where each lane of a 4X port runs a bit rate of 25Gb/s with 64b/66b encoding, resulting in an effective bandwidth of 100Gb/s.
HDR is an InfiniBand data rate, where each lane of a 4X port runs a bit rate of 50Gb/s with 64b/66b encoding, resulting in an effective bandwidth of 200Gb/s.
The RS232 serial “Console” port is labeled
.
The port labeled “Console” is an RS232 serial port on the front side of the chassis that is used for initial configuration and debugging. Upon first installation of the system, you need to connect a PC to this interface and configure network parameters for remote connections. Refer to Configuring Network Attributes to view the full procedure.
This interface is not found in externally managed systems.
The RJ45 Ethernet “MGT” port is labeled
.
The RJ45 Ethernet port provides access for remote management. The management port is configured with auto-negotiation capabilities by default (100MbE to 1000GbE). The management port's network attributes (such as IP Address) must be pre-configured via the RS232 serial console port or by DHCP before use. Please refer to Configuring Network Attributes to view the full procedure.
Make sure you use only FCC compliant Ethernet cables.
The micro USB interface located in the system's front side is USB 2.0 compliant and can be used by MLNX-OS software to connect to an external disk for software upgrade or file management. The connector comes in a standard micro USB shape. To view the full matrix of micro USB configuration options, refer to Management Interfaces, PSUs and Fans.
USB 1.0 is not supported.
Do not use excessive force when inserting or extracting the USB disk to and from the connector.
The I²C connector is combined with the Console connector and is located on the front side of the system (the RJ45 connector). It can be used with the HAR255 I²C DB9 splitting harness.
This interface is not found in externally managed systems.
The I²C interface is used for debugging and is intended for NVIDIA debug personnel only.
The reset button is located on the front side of the system under the RJ45 connector. This reset button requires a tool to be pressed.
Do not use a sharp pointed object such as a needle or a push pin for pressing the reset button. Use a flat object to push the reset button.
To reset the system and the CPU of its management board, push the Reset button and keep it pressed for up to 15 seconds.
To reset the system, the CPU of its management board and the “admin” password, push the Reset button and keep it pressed for at least 15 seconds. You will then be able to enter without a password and set a new password for the user “admin”. In the externally managed systems, the Reset button resets the device.
See LED Notifications.