NVIDIA MLNX_OFED Documentation v5.8-4.1.5.0 LTS
Linux Kernel Upstream Release Notes v5.17

Changes and New Features

The following are the new features and changes that were added in this version. The supported adapter cards are specified as follows:

Supported Cards

Description

All HCAs

Supported in the following adapter cards unless specifically stated otherwise:

ConnectX-4 / ConnectX -4 Lx / ConnectX-5 / ConnectX-6 / ConnectX-6 Dx / ConnectX-6 Lx / ConnectX-7 / BlueField-2

ConnectX-6 Dx and above

Supported in the following adapter cards unless specifically stated otherwise:

ConnectX-6 Dx / ConnectX-6 Lx / ConnectX-7 / BlueField-2

ConnectX-6 and above

Supported in the following adapter cards unless specifically stated otherwise:

ConnectX-6 / ConnectX-6 Dx / ConnectX-6 Lx / ConnectX-7 / BlueField-2

ConnectX-5 and above

Supported in the following adapter cards unless specifically stated otherwise:

ConnectX-5 / ConnectX-6 / ConnectX-6 Dx / ConnectX-6 Lx / ConnectX-7 / BlueField-2

ConnectX-4 and above

Supported in the following adapter cards unless specifically stated otherwise:

ConnectX-4 / ConnectX -4 Lx / ConnectX-5 / ConnectX-6 / ConnectX-6 Dx / ConnectX-6 Lx / ConnectX-7 / BlueField-2

Feature/Change

Description

5.8-4.1.5.0

Operating Systems

Added support for the following operating systems:

  • RHEL 9.3

  • RHEL 8.9

  • Debian 10.13

Bug Fixes

Bug Fixes in This Version

5.8-3.0.7.0

Operating Systems

Added support for the following operating systems:

  • RHEL 9.2

  • RHEL 8.8

  • SLES15-SP5

  • KYLIN10-SP3

General

Bug fixes

5.8-2.0.3.0

General

Bug fixes

5.8- 1.1.2.1

General

Bug fixes

5.8- 1.0.1.1

Remove Dependency Between SR-IOV and eSwitch Mode

[All HCAs] Removed dependency between SR-IOV and eSwitch mode. Currently, there are three eSwitch modes: none, legacy, and switchdev (non of which are the default mode). When disabling SR-IOV, the current eSwitch mode will be changed to none. This feature removes eSwitch mode none and also removes dependency between SR-IOV and eSwitch mode.

DevLink Parallel Command

[All HCAs] Added support for running DevLink commands in parallel on different DevLink devices is possible. For example, burning firmware on a few cards on the same host in parallel using DevLink API is now possible.

Graceful Shutdown of Parent and Page Supplier

[All HCAs] Set default graceful period values for functions based on their type. ECPFs will get graceful period of 3 minutes, PFs get 1 minute, and VFs/SFs get 30 seconds.

N Pulses Per Second (NPPS)

[ConnectX-6 Dx and above] Enhanced NPPS to allow setting a pulse period higher than 1 pulse per second and to allow setting the pulse width.

If the width is unset, the driver implicitly sets it to half the given period (the width should be less than the pulse period). In this release, the pulse duration ranges between 65536 NS–524288 NS.

Remote Invalidate Option for MKeys

[All HCAs] Addded support for the option to enable remote invalidation when creating a new mkey. This way the rkey for a memory region can be changed frequently.

GPUDirect Over DMA-BUF

[All HCAs] Added support for GPUDirect support over dma-buf. As such, using the new mechanism nv_peer_mem is no longer required.

The following is required for dma-buf support:

  • Linux kernel version 5.12 or later

  • OpenRM version 515 or later

Perftest support was added as well:

Default option in perftest is without dmabuf. To run with this option, add --use_cuda_dmabuf in addition to use_cuda flag.

Floating LID

[ConnectX-7] Added support for Floating LID (FLID) which can be used to identify a group of InfiniBand routers that allow communication with another subnet's entity. With this feature, multiple routers can be used per destination so that adaptive routing is supported.

The FLID feature needs support from components such as the host, the subnet manager, the router, and more. This feature is only supported on the host portion of the sysrem.

General

Bug fixes

For additional information on the new features, please refer to MLNX_OFED User Manual.

Customer Affecting Change

Description

5.8-2.0.3.0

Installation, ISO, RedHat

In order to address RHEL kernel symbol changes, starting from OFED version 5.8-2.0.3.0, ISO images for the following operating systems are built with the updated kernel versions as follows:

Copy
Copied!
            

OS Name     Old Kernel     New Kernel rhel8.6-aarch64    4.18.0-372.9.1.el8_6.aarch64     4.18.0-372.41.1.el8_6.aarch64  rhel8.6-ppc64le    4.18.0-372.0.1.el8_6.ppc64le     4.18.0-372.41.1.el8_6.ppc64le  rhel8.6-x86_64    4.18.0-372.9.1.el8_6.x86_64     4.18.0-372.41.1.el8_6.x86_64 rhel8.7-aarch64    4.18.0-425.3.1.el8.aarch64     4.18.0-425.14.1.el8_7.aarch64 rhel8.7-ppc64le    4.18.0-425.3.1.el8.ppc64le     4.18.0-425.14.1.el8_7.ppc64le rhel8.7-x86_64     4.18.0-425.3.1.el8.x86_64     4.18.0-425.14.1.el8_7.x86_64 rhel9.0-aarch64    5.14.0-70.13.1.el9_0.aarch64     5.14.0-70.46.1.el9_0.aarch64  rhel9.0-ppc64le    5.14.0-70.13.1.el9_0.ppc64le     5.14.0-70.46.1.el9_0.ppc64le  rhel9.0-x86_64    5.14.0-70.13.1.el9_0.x86_64     5.14.0-70.46.1.el9_0.x86_64 rhel9.1-aarch64    5.14.0-162.6.1.el9_1.aarch64     5.14.0-162.19.1.el9_1.aarch64  rhel9.1-ppc64le    5.14.0-162.6.1.el9_1.ppc64le     5.14.0-162.19.1.el9_1.ppc64le  rhel9.1-x86_64    5.14.0-162.6.1.el9_1.x86_64     5.14.0-162.19.1.el9_1.x86_64

This change comes to support RedHat updated kernels without the need to add --add-kernel-support during OFED installation.

MLNX_OFED Verbs API Migration

As of MLNX_OFED v5.0 release (Q1 of the year 2020), MLNX_OFED Verbs API have migrated from the legacy version of user space verbs libraries (libibervs, libmlx5, etc.) to the Upstream version rdma-core.

For the list of MLNX_OFED verbs APIs that have been migrated, refer to Migration to RDMA-Core document.

© Copyright 2023, NVIDIA. Last updated on Dec 28, 2023.