InfiniBand Related Troubleshooting

Linux Kernel Upstream Release Notes v6.5

Issue

Cause

Solution

No link over NVIDIA® ConnectX®-6 IB VF.

Old OpenSM version.

Use UFM Appliance version 4.0 and above as it automatically installs OpenSM v5.4.0.

For further information on how to add support for additional devices, please refer to UFM Appliance User Manual.

The InfiniBand interfaces are not up after the first reboot after the installation process is completed.

Port status might be PORT_DOWN:

Switch port state might be “disabled” or cable is disconnected.

Enable switch admin or connect cable.

Port status might be PORT_INITIALIZED:

SM might not be running on the fabric.

Run the SM on the fabric.

Port status might be PORT_ARMED:

Firmware issue.

Please contact Support.

SR-IOV might be enabled with firmware that does not support SR-IOV and IPoIB simultaneously.

In this case, the driver will report an error message stating that IPoIB is not supported by the firmware.

Use the mlxconfig tool to disable SR-IOV.

Consult the MFT User Manual for further details.

© Copyright 2023, NVIDIA. Last updated on Nov 1, 2023.