Interfaces

QM87xx 1U HDR 200Gb/s InfiniBand Switch Systems User Manual

The systems support the following interfaces:

  • Data interfaces - InfiniBand

  • 10/100/1000Mb Ethernet management interface (RJ45)*

  • USB port (uUSB connector)

  • RS232 Console port (RJ45)*

  • I²C interface*

  • Reset button

  • Status and Port LEDs

*This interface is not found in externally managed systems.

In order to review the full configuration options matrix, refer to Management Interfaces, PSUs and Fans.

The data interfaces use QSFP56 connectors. The full list of interfaces per system is provided in Speed and Switching Capabilities.

Each QSFP56 port can be connected with QSFP56 cable or connector for 40/56/100/200 Gb/s. The system offers Class 7 (5W) QSFP56 transceivers support.

Speed

InfiniBand speed is auto-adjusted by the InfiniBand protocol. Mellanox systems support QDR/FDR/EDR/HDR InfiniBand.

  • FDR is an InfiniBand data rate, where each lane of a 4X port runs a bit rate of 14.0625Gb/s with 64b/66b encoding, resulting in an effective bandwidth of 56.25Gb/s.

  • EDR is an InfiniBand data rate, where each lane of a 4X port runs a bit rate of 25Gb/s with 64b/66b encoding, resulting in an effective bandwidth of 100Gb/s.

  • HDR is an InfiniBand data rate, where each lane of a 4X port runs a bit rate of 50Gb/s with 64b/66b encoding, resulting in an effective bandwidth of 200Gb/s.

Warning

The RS232 serial “Console” port is labeled

image2019-3-5_16-50-5-version-1-modificationdate-1704381633383-api-v2.png

.

The Console port is an RS232 serial port on the back side of the chassis that is used for initial configuration and debugging. Upon first installation of the system, you need to connect a PC to this interface and configure network parameters for remote connections. Refer to Configuring Network Attributes to view the full procedure.

Warning

This interface is not found in externally managed systems.

Warning

The RJ45 Ethernet “MGT” port is labeled

image2019-3-5_16-52-4-version-1-modificationdate-1704381632960-api-v2.png

.

The Management RJ45 Ethernet ports provide access for remote management. The management ports are configured with auto-negotiation capabilities by default (100MbE to 1000GbE). The management ports’ network attributes (such as IP Address) need to be pre-configured via the RS232 serial console port or by DHCP before use. Refer to Configuring Network Attributes to view the full procedure.

Warning

This interface is not found in externally managed systems.

Important

Make sure you use only FCC compliant Ethernet cables.

The USB interface is USB 2.0 compliant and can be used by MLNX-OS software to connect to an external disk for software upgrade or file management. The connector comes in a standard micro USB shape. To view the full matrix of micro USB configuration options, refer to Management Interfaces, PSUs and Fans.

Warning
  • USB 1.0 is not supported.

  • Do not use excessive force when inserting or extracting the USB disk to and from the connector.

The I²C connector is combined with the Console connector and is located on the front side of the system (the RJ45 connector). It can be used with the I²C DB9 to RJ45 splitting harness.

Warning
  • This interface is not found in managed systems. It is available in QM8790 systems only.

  • Apart from the initial configuration, I²C interface is made exclusively for debugging and troubleshooting. Only FAEs are authorized to connect through it.

Important

Only original Mellanox cables supplied with the switch package can be used to connect a switch system to the server.

Connecting any cable other than the Mellanox supplied console cable may cause an I²C hang. Using uncertified cables may damage the I²C interface.

Refer to the Replacement Parts Ordering Numbers appendix for harness details.

The reset button is located on the front side of the system next to the fan status LEDs. This reset button requires a tool to be pressed.

Important

Do not use a sharp pointed object such as a needle or a push pin for pressing the reset button. Use a flat object to push the reset button.

  • To reset the system, push the reset button for less than 15 seconds.

  • When using an Onyx (MLNX-OS) based system, keeping the reset button pressed for more than 15 seconds will reset the system and the “admin” password, this should allow you to enter without a password and set a new password for the user “admin”.

© Copyright 2023, NVIDIA. Last updated on Jan 7, 2024.