Ethernet Related Issues

Ethernet interfaces renaming fails leaving them with names such as renameXY.

Invalid udev rules.

Review the udev rules inside the "/etc/ udev/rules.d/70-persistent-net.rules" file. Modify the rules such that every rule is unique to the target interface, by adding correct unique attribute values to each interface, such as dev_id, dev_port and KERNELS or address).

Example of valid udev rules:

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*",

ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*",

ATTR{dev_port}=="0", KER- NELS=="0000:08:00.0", NAME="eth4" SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*",

ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*",

ATTR{dev_port}=="1", KER- NELS=="0000:08:00.0", NAME="eth5"

No link.

Misconfiguration of the switch port or using a cable not supporting link rate.

• Ensure the switch port is not down

• Ensure the switch port rate is configured to the same rate as the adapter's port

Degraded performance is measured when having a mixed rate environment (10GbE, 40GbE and 56GbE).

Sending traffic from a node with a higher rate to a node with lower rate.

Enable Flow Control on both switch ports and nodes:

• On the server side run:
  ethtool -A <interface> rx on tx on

• On the switch side run the following command on the relevant interface:
  send on force and receive on force

No link with break-out cable.

Misuse of the break-out cable or misconfiguration of the switch's split ports

• Use supported ports on the switch with proper configuration. For further information, please refer to the MLNX_OS User Manual.

• Make sure the QSFP breakout cable side is connected to the SwitchX.

Physical link fails to negotiate to maximum supported rate.

The adapter is running an outdated firmware.

Install the latest firmware on the adapter.

Physical link fails to come up while port physical state is Polling.

The cable is not connected to the port or the port on the other end of the cable is disabled.

• Ensure that the cable is connected on both ends or use a known working cable

• Check the status of the connected port using the ibportstate command and enable it if necessary

Physical link fails to come up while port physical state is Disabled.

The port was manually disabled.

Restart the driver:

/etc/init.d/openibd restart