QM9700/QM9790 1U NDR 400Gb/s InfiniBand Switch Systems User Manual
QM9700/QM9790 1U NDR 400Gb/s InfiniBand Switch Systems User Manual

Cable Installation

In some switch models, the product's package includes cable retainers. It is highly recommended to use them in order to secure the power cables in place.

When installing retainers for the PSUs of the QM97x0 switch systems, please adhere to the following instructions:

  1. Verify the integrity of the retainer assembly, as demonstrated in the below table:- The snaps' push-pins must have visible edges with no broken or torn parts.
    - The shoulders' pins should be in-tact and must not be bent inwards.

    Proper Condition

    Improper Condition

    image2021-3-8_13-43-33-version-1-modificationdate-1711027494827-api-v2.png

    image2021-3-8_13-42-12-version-1-modificationdate-1711027495160-api-v2.png

  2. It is advised to place the PSU on a flat, stable surface. While you secure the PSU in place, use two thumbs to insert the retainer's two snaps into the designated holes located near the AC inlet. Make sure that the retainer's plastic loop is facing upwards, as demonstrated in the below table.

    Warning

    For demonstration purposes, the images in this document show C2P (Connector-to-Power) airflow PSUs with red latches, yet the instructions apply to P2C (Power-to-Connector) PSUs with blue latches as well.

    Correct Insertion

    Incorrect Insertion

    image2022-2-23_14-59-39-version-1-modificationdate-1711027492717-api-v2.png

    image2022-2-23_14-58-45-version-1-modificationdate-1711027492377-api-v2.png

  3. Push the retainer until the shoulders' pins (in blue circles below) are open and aligned with the PSU front panel, as shown in the following table:

    Fully Mated Retainer

    image2022-2-23_15-1-16-version-1-modificationdate-1711027493307-api-v2.png

    image-2024-1-17_16-0-59-version-1-modificationdate-1711032992266-api-v2.png

  4. Make sure that the retainer is fully locked in place by gently attempting to pull it outwards.

  5. Open the plastic loop and route the AC cord through it. Locate the loop over the AC cord, as shown in the following table, and fasten it tightly.

    Proper Loop Placement

    Improper Loop Placement

    image2022-2-23_15-9-13-version-1-modificationdate-1711027493617-api-v2.png

    image2022-2-23_15-9-56-version-1-modificationdate-1711027493967-api-v2.png

Warning

Each cable retainer can be used once only. Once the retainer has been fully inserted and the shoulders' pins have been adjusted, the retainer cannot be used again, and should be discarded if pulled out.

All cables can be inserted or removed with the unit powered on.

To insert a cable, press the connector into the port receptacle until the connector is firmly seated. The LED indicator, corresponding to each data port, will light when the physical connection is established. When a logical connection is made, the relevant port LED will turn on.

To remove a cable, disengage the locks and slowly pull the connector away from the port receptacle. The LED indicator for that port will turn off when the cable is unseated.

For full cabling guidelines, ask your NVIDIA representative for a copy of NVIDIA Cable Management Guidelines and FAQs Application Note.

For more information about port LEDs, refer to Port LEDs.

Important

Do not force the cable into the cage with more than 40 newtons/9.0 pounds/4kg force. Greater insertion force may cause damage to the cable or to the cage.

Warning

Unused OSFP cages must be closed with the thermal caps supplied with the system.

image2021-12-1_11-35-14-version-1-modificationdate-1711027497193-api-v2.png

Cable Orientation

image2021-12-1_11-2-31-version-1-modificationdate-1711027496257-api-v2.png

image2021-12-1_10-57-20-version-1-modificationdate-1711027495453-api-v2.png

image2021-12-1_11-3-22-version-1-modificationdate-1711027496460-api-v2.png

image2021-12-1_10-57-20-version-1-modificationdate-1711027495453-api-v2.png

Splitter (Breakout) Cables and Adapters

In the QM9700 and QM9790 systems, a single OSFP cage contains 2 NDR ports, and a single NDR port (quad-lane) is divided into 2 dual-lane ports. This maximizes flexibility by enabling end users to use a combination of dual-lane and quad-lane interfaces according to the specific requirements of their network. For the systems splitting options, see QM9700/QM9790 Splitting Options below.

Splitting a port changes the notation of that port from x/y/z to x/y/z/i, with “x/y/z” indicating the previous notation of the port prior to the split, and “i” indicating the number of the resulting single-lane port (1,2). Each sub-physical port is then handled as an individual port. For example, splitting port 1/5/1 into 2 lanes results in ports 1/5/1/1 and 1/5/1/2. For full notation schematics, see Port Notation Schematics.

Warning

The following behavior should be expected when disconnecting a 1:2 splitter cable (from cages in both the upper and lower rows):

  • When you disconnect a cable marked as “1”, the CLI <cage number>/1 will always go down, and the left LED of the cage will be turned off.

  • When you disconnect the cable marked as “2”, the CLI <cage number>/2 will always go down, and the right LED of the cage will be turned off.

Breakout Cable Example

image2021-11-3_14-30-54-version-1-modificationdate-1711027509773-api-v2.png

Warning

This feature is available only in Quantum/Quantum-2 based systems.

Tip
  • Splitting the interface deletes all configuration on that interface.

  • In order to be able to use this feature, the system profile command must be activated with split-ready configuration (cross-reference to system profile command).

  • Changes will take effect after reset. In order to reset an unmanaged switch, please reboot the switch, or run flint -d <device> swreset.

For more information on how to change the system’s profile to allow Split-Ready configuration, how to change the module type to a split mode, and how to unsplit a split port when using QM9700, please refer to the "InfiniBand Switching" chapter in the latest MLNX-OS® User Manual. For QM9790, please refer to latest NVIDIA Firmware Tools (MFT) Documentation.

QM9700/QM9790 Splitting Options

image2023-4-30_19-10-51-version-1-modificationdate-1711027486960-api-v2.png

image2019-1-22_18-30-8-version-1-modificationdate-1711027502303-api-v2.png

All NDR ports are splittable. Each OSFP cage contains two NDR ports of 400G, and each NDR port can be split to two.

Port Notation Schematics

Two port notation profiles can be selected for the QM97x0 NDR switch systems. In both cases, each cage in the system's front panel holds two ports from the same ASIC, and the cage numbers are global:

  1. ASIC/Cage/Port:

    image2021-11-4_9-28-18-version-1-modificationdate-1711027511560-api-v2.png

  2. ASIC/Cage/Port/Split:

    image2021-11-4_9-27-11-version-1-modificationdate-1711027511337-api-v2.png

Logical Port Numbering Schematic

Two profiles can be selected for the QM97x0 NDR switch systems. The first one defines the system as a pure 64-port NDR (32 cages) switch. The other profile permits any or all NDR ports to be split into two 2X (NDR200) ports. The following diagrams attempt to show how the logical ports map onto the physical NDR ports, as viewed by the IB tools (e.g. ibnetdiscover):

Switch Profile: Non-Splittable (Suitable for L2/Spine Switches)

image2023-4-30_19-29-56-version-1-modificationdate-1711027487673-api-v2.png

Warning

The IB tools report 65 logical ports. Port 65 is an internal port used for the SHARP Aggregation Node when SHARP is enabled.

Switch Profile: Splittable

image-2024-3-21_15-29-57-version-1-modificationdate-1711027796090-api-v2.png

Warning

Note: The IB tools will report 129 logical ports. Port 129 is an internal port used for the SHARP Aggregation Node when SHARP is enabled.


© Copyright 2024, NVIDIA. Last updated on Mar 21, 2024.