NVIDIA MLNX_OFED Documentation v23.10-2.1.3.1 LTS
NVIDIA MLNX_OFED Documentation v23.10-2.1.3.1 LTS

Ethernet Related Issues

Issue

Cause

Solution

Ethernet interfaces renaming fails leaving them with names such as renameXY.

Invalid udev rules.

Review the udev rules inside the "/etc/ udev/rules.d/70-persistent-net.rules" file. Modify the rules such that every rule is unique to the target interface, by adding correct unique attribute values to each interface, such as dev_id, dev_port and KERNELS or address).

Example of valid udev rules:

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*",

ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*",

ATTR{dev_port}=="0", KER- NELS=="0000:08:00.0", NAME="eth4" SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*",

ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*",

ATTR{dev_port}=="1", KER- NELS=="0000:08:00.0", NAME="eth5"

No link.

Misconfiguration of the switch port or using a cable not supporting link rate.

  • Ensure the switch port is not down

  • Ensure the switch port rate is configured to the same rate as the adapter's port

Degraded performance is measured when having a mixed rate environment (10GbE, 40GbE and 56GbE).

Sending traffic from a node with a higher rate to a node with lower rate.

Enable Flow Control on both switch ports and nodes:

  • On the server side run:

    ethtool -A <interface> rx on tx on

  • On the switch side run the following command on the relevant interface:

    send on force and receive on force

No link with break-out cable.

Misuse of the break-out cable or misconfiguration of the switch's split ports

  • Use supported ports on the switch with proper configuration. For further information, please refer to the MLNX_OS User Manual.

  • Make sure the QSFP breakout cable side is connected to the SwitchX.

© Copyright 2023, NVIDIA. Last updated on Feb 29, 2024.