NVIDIA MLNX_OFED Documentation v5.6-1.0.3.3
Linux Kernel Upstream Release Notes v5.17

Changes and New Features

The following are the new features and changes that were added in this version. The supported adapter cards are specified as follows:

Supported Cards

Description

All HCAs

Supported in the following adapter cards unless specifically stated otherwise:

ConnectX-4 / ConnectX -4 Lx / ConnectX-5 / ConnectX-6 / ConnectX-6 Dx / ConnectX-6 Lx / ConnectX-7 / BlueField-2

ConnectX-6 Dx and above

Supported in the following adapter cards unless specifically stated otherwise:

ConnectX-6 Dx / ConnectX-6 Lx / ConnectX-7 / BlueField-2

ConnectX-6 and above

Supported in the following adapter cards unless specifically stated otherwise:

ConnectX-6 / ConnectX-6 Dx / ConnectX-6 Lx / ConnectX-7 / BlueField-2

ConnectX-5 and above

Supported in the following adapter cards unless specifically stated otherwise:

ConnectX-5 / ConnectX-6 / ConnectX-6 Dx / ConnectX-6 Lx / ConnectX-7 / BlueField-2

ConnectX-4 and above

Supported in the following adapter cards unless specifically stated otherwise:

ConnectX-4 / ConnectX -4 Lx / ConnectX-5 / ConnectX-6 / ConnectX-6 Dx / ConnectX-6 Lx / ConnectX-7 / BlueField-2

Feature/Change

Description

5.6-1.0.3.3

General

New Adapter Card Support

Added support for ConnectX-7 adapter cards. ConnectX-7 has the same feature set as the ConnectX-6 adapter card.

ASAP2 Features

Bridge Spoof Check

[All HCAs] Added support for spoof check with TC flower rules on representors attached to bridge to mirror spoof check SR-IOV functionality.

Setting VF Group Rate Limit

[ConnectX-5 and above] Added support for setting VF group rate limit using Devlink command.

TC Flows on Shared Block

[ConnectX-5 and above] Added support for creation of TC flows on shared block of VF representors.

Flow Metering

[ConnectX-6 Dx and above] Added support for offloading OpenFlow Meters in OVS-DPDK.

Please note the following:

  • Meter offload can be applied only on port 0 and it's VFs

  • Only one meter per flow is allowed

  • Only one meter band per meter is allowed

  • Only meter band type drop is supported

  • Meter-stats might not be accurate

Core Features

Firmware Reset

[BlueField-2] Added support of firmware reset in DPU NIC mode.

Increased Robustness of mlx5_core Driver Recovery

[All HCAs] Increased the firmware pre-initialization timeout from 2 minutes to 2 hours when waiting for firmware during driver health recovery, allowing the driver to passively recover from a firmware reset, even if the reset takes an unusually long time. Additionally, added an exit clause to the wait for firmware loop, allowing immediate response to a user initiated device removal.

NetDev Features

Ethtool CQE Mode Control

[ConnectX-4 and above] Replaced the vendor-specific Ethtool API (priv-flag) with a standard Ethtool API (replaced 'ethtool --set-priv-flags ethX rx_cqe_moder on/off tx_cqe_moder on/off' with 'ethtool -C ethX cqe-mode-rx on/off cqe-mode-tx on/off'). This decreases the amount of vendor-specific configurations and aligns mlx5 driver with the upstream Ethtool API.

SyncE

[ConnectX-6 Dx] Added an indication in SyncE Daemon that states whether SyncE engine moved to holdover state due to failure (the reason for failure will be displayed). In addition, added indication whether SyncE engines collected enough frequency samples in order to move to holdover.

RDMA Features

VFIO, CQ Interrupt Mode

[ConnectX-5 and above] Added support for VFIO applications to listen on and capture completion events via the Event Queue mechanism.

VFIO, Asynchronous Event

[ConnectX-5 and above] Added support for VFIO applications to listen on and capture device asynchronous events via the Event Queue mechanism.

Security

OVS-IPSec Full Offload

[BlueField-2] Added support for configuration of IPsec full offload using OVS by adding VXLAN tunnel to OVS with the PSK option.

Software Steering Features

Full Tunnel Header Matching

[ConnectX-6 Dx and above] Added support for using full-tunnel-header matching along with many other criteria within one matcher. This feature uses the new definer index, defined in the firmware, to build a matcher so that the full tunnel header matching can be used along with all other criteria.

Matching Granularity Change

[ConnectX-5 and above] Added support for matching granularity change. As a result, when creating FDB flow with destination of VPORT, a src_port matching must no longer be added. Now, FDB flow can match all vports and goto a VPORT destination. The new behavior is the same as done on firmware steering.

Installation

Installation

New options were added to the ofed_uninstall.sh script: --only-kernel and --only-user. Those can be used to uninstall only kernel packages or only user-space packages (the equivalent of kernel-only install or user-space-only install, respectively). This may be useful to keep different sets of kernel and user-space installations.

When running the uninstall script with a combination of --only-kernel and --only-user produced an undefined result.

For additional information on the new features, please refer to MLNX_OFED User Manual.

Customer Affection Change

Description

5.6-1.0.3.3

Interface Renaming, PF/VF, Udev

The OFED driver no longer performs Ethernet NetDev interface renaming for PFs and VFs.

The udev rules file which implemented renaming (82-net-setup-link.rules) and its supporting script vf-net-link-name.sh are no longer installed by default.

Renaming is thus performed by underlying mechanisms -- in udev, in the kernel, and in the BIOS.

Users who wish to continue using the OFED driver renaming mechanism must add option —copy-ifnames-udev to the OFED install command.

To install these files at a later time, copy them from one of the following directories:

  • /usr/share/doc/mlnx-ofa_kernel (RHEL8 and newer)

  • /usr/share/doc/mlnx-ofa_kernel-[1-9]* (RHEL 7.X)

  • /usr/share/doc/packages/mlnx-ofa_kernel (SLES)

  • /usr/share/doc/mlnx-ofed-kernel-utils/examples (Debian-based releases)

Warning
  • File 82-net-setup-link.rules should be copied to directory /etc/udev/rules.d

  • File vf-net-link-name.sh should be copied to directory /etc/infiniband (make sure that it has both read and execute permission)

  • After copying over the files, the driver should be restarted for the copied files to take effect

  • Customers who wish prevent renaming of NetDev names should add "net.ifnames=0 biosdevname=0" to the kernel boot command line, and then reboot the host

Community Operating Systems

Starting OFED 5.6, NVIDIA is introducing a new support model for OFED used on open source community operating systems. The goal of this new support model is to enable customers to use community-maintained variants of the Linux operating system, without being limited to major distributions that NVIDIA provides primary support for. For more information, see "Installation on Community Operating Systems" section in the user manual. For a list of supported Community OSs, please see "Supported Community Operating Systems" section in the release notes.

ar_mgr Subnet Manager Plugin

ar_mgr subnet manager plugin is no longer supported.

For adaptive routing and SHIELD subnet manager configuration, please see the MLNX_OFED user manual.

Fabric Collector in UFM

Starting UFM v6.7, Fabric Collector is no longer supported. For more information, see the UFM release notes.

Warning

Please note that UFM is no longer bundled with OFED.

OVS-DPDK—Partial Offload

Starting OFED 5.6, OVS-DPDK does not support partial offload.

MLNX_OFED Verbs API Migration

As of MLNX_OFED v5.0 release (Q1 of the year 2020), MLNX_OFED Verbs API have migrated from the legacy version of user space verbs libraries (libibervs, libmlx5, etc.) to the Upstream version rdma-core.

For the list of MLNX_OFED verbs APIs that have been migrated, refer to Migration to RDMA-Core document.

© Copyright 2023, NVIDIA. Last updated on Nov 27, 2023.