Changes and New Features History
This section includes history of changes and new feature of three major (GA) releases back. For older versions' history, please refer to their dedicated release notes.
Supported Cards | Description |
All HCAs | Supported in the following adapter cards unless specifically stated otherwise: ConnectX-4 / ConnectX -4 Lx / ConnectX-5 / ConnectX-6 / ConnectX-6 Dx / ConnectX-6 Lx / ConnectX-7 / BlueField-2 |
ConnectX-6 Dx and above | Supported in the following adapter cards unless specifically stated otherwise: ConnectX-6 Dx / ConnectX-6 Lx / ConnectX-7 / BlueField-2 |
ConnectX-6 and above | Supported in the following adapter cards unless specifically stated otherwise: ConnectX-6 / ConnectX-6 Dx / ConnectX-6 Lx / ConnectX-7 / BlueField-2 |
ConnectX-5 and above | Supported in the following adapter cards unless specifically stated otherwise: ConnectX-5 / ConnectX-6 / ConnectX-6 Dx / ConnectX-6 Lx / ConnectX-7 / BlueField-2 |
ConnectX-4 and above | Supported in the following adapter cards unless specifically stated otherwise: ConnectX-4 / ConnectX -4 Lx / ConnectX-5 / ConnectX-6 / ConnectX-6 Dx / ConnectX-6 Lx / ConnectX-7 / BlueField-2 |
Feature/Change | Description |
23.10-4.0.9.1 | |
General | |
Embedded Components | Updated the versions of the following embedded component:
For further information, see Embedded Components section. |
General | |
Feature/Change | Description |
23.10-3.2.2.0 | |
Operating Systems | Added support for the following OSes:
For further information, see Supported Operating Systems section |
Embedded Components | Updated the versions of the following embedded component:
For further information, see Embedded Components section. |
Bridge Offload | Enabled Bridge Offload in mlx5_core by default. A new option was added to the configure of mlnx_ofa_kernel to disable it: To use this option from |
General | |
5.9-0.5.6.0 | |
ASAP2 Features | |
Linux Bridge VLAN Filtering of 802.1 Q Packets | [ConnectX-6 Dx] Extended mlx5 Linux bridge VLAN offload to support packets tagged with 802.1 Q VLAN ethertype. |
Offloading sFlow Sampling Rules | [ConnectX-5 and above] Added support for sFlow sampling rules offloads. sFlow is an industry standard technology for monitoring high speed switched networks. Open vSwitch integrated sFlow to extend the visibility into virtual servers, ensuring data center visibility and control. |
Core Features | |
Configuring Shared Buffer Size | [ConnectX-6 Dx and above] Enabled user to control shared buffer size and configuration, implicitly. As with each port buffer command the user triggers, the shared buffer configuration will be updated accordingly by the driver. |
Control SF Class | [All HCAs] Added support for Control SF Class. Each PCI, PF, VF, SF function, by default, has netdevice, RDMA, and vdpa-net devices always enabled. This feature enables the user to control which device functionality to enable/disable. Note: Requires kernel 5.18 or higher. |
NetDev Features | |
Support RSS over XSK Queues | [All HCAs] Use default RSS functionality to spread traffic across different XSK queues instead of having to provide explicit steering rules. |
TLS TIS Pool | [TLS-Enabled Devices] Per-connection hardware TIS objects is used to maintain the device TLS TX context. Use a SW TIS pool for recycling the TIS objects instead of destroying/creating them. This reduces the interaction with the device via the FW command interface, which increases the TLS connection rate. |
RDMA Features | |
Expand Rep Counters | [ConnectX-5 and above] Adding RDMA traffic-only counters for rep devices. These counters can now be read from host with ethtool or from sysfs and not only from the cointainer. |
UMR QP Recilency | [ConnectX-5 and above] Added a recovery flow for the driver's UMR logic so that other UMR requests can be proccessed after the error UMR was dropped and the UMR QP was reset. Previously, a faulty UMR request would have moved the QP to error state and disable any option to continue issuing UMRs. |
General | |
Feature/Change | Description |
5.8- 1.1.2.1 | |
General | |
5.8- 1.0.1.1 | |
Remove Dependency Between SR-IOV and eSwitch Mode | [All HCAs] Removed dependency between SR-IOV and eSwitch mode. Currently, there are three eSwitch modes: none, legacy, and switchdev (non of which are the default mode). When disabling SR-IOV, the current eSwitch mode will be changed to none. This feature removes eSwitch mode none and also removes dependency between SR-IOV and eSwitch mode. |
DevLink Parallel Command | [All HCAs] Added support for running DevLink commands in parallel on different DevLink devices is possible. For example, burning firmware on a few cards on the same host in parallel using DevLink API is now possible. |
Graceful Shutdown of Parent and Page Supplier | [All HCAs] Set default graceful period values for functions based on their type. ECPFs will get graceful period of 3 minutes, PFs get 1 minute, and VFs/SFs get 30 seconds. |
N Pulses Per Second (NPPS) | [ConnectX-6 Dx and above] Enhanced NPPS to allow setting a pulse period higher than 1 pulse per second and to allow setting the pulse width. If the width is unset, the driver implicitly sets it to half the given period (the width should be less than the pulse period). In this release, the pulse duration ranges between 65536 NS–524288 NS. |
Remote Invalidate Option for MKeys | [All HCAs] Added support for the option to enable remote invalidation when creating a new mkey. This way the rkey for a memory region can be changed frequently. |
GPUDirect Over DMA-BUF | [All HCAs] Added support for GPUDirect support over dma-buf. As such, using the new mechanism nv_peer_mem is no longer required. The following is required for dma-buf support:
Perftest support was added as well: Default option in perftest is without dmabuf. To run with this option, add --use_cuda_dmabuf in addition to use_cuda flag. |
General | |
Feature/Change | Description |
23.10-2.1.3.1 | |
General | |
Feature/Change | Description |
5.7-1.0.2.0 | |
Support Representor Metering Over SFs | [ConnectX-6 Dx and above and BlueField-2] Extended the support of representor metering from supporting only VFs representor to also supporting SFs representor. |
Exposing Error Counters on a VPort Manager | [ConnectX-4 and above] Added support for exposing error counters on a VPort manager function for all other VPorts. These counters can be used to detect malicious users who are exploiting flows that can slow the device. The counters are exposed through debugfs under: /sys/kernel/debug/mlx5/esw/<func>/vnic_diag/ |
Memory Consumption Minimization | [ConnectX-4 and above] Added support for providing knobs which enable users to minimize memory consumption of mlx5 functions (PF/VF/SF). |
XDP Support for Uplink Representors | [ConnectX-5 and ConnectX-6 Dx) Added XDP support for uplink representors in switchdev mode. |
Resiliency to tx_port_ts | [ConnectX-6 Dx and above] Added resiliency to the tx_port_ts feature. private-flag may be enabled via ethtool tx_port_ts which provides a more accurate time-stamp. In very rare cases, the said time-stamp was lost, leading to losing the synchronization altogether. This feature allows for fast recovery and allows to quickly regain synchronization. |
Database of Devlink Health Asserts | [ConnectX-4 and above] Health buffer now contains more debug information like the epoch time in sec of the error and the error's severity. The print to dmesg is done with the debug level corresponding to the error's severity. This allows the user to use dmesg attribute: dmesg --level to focus on different severity levels of firmware errors. |
Expose FEC Counters via Ethtool | [ConnectX-5 and above] Exposed the following FEC (forward error detection) counters: ETHTOOL_A_FEC_STAT_CORRECTED
ETHTOOL_A_FEC_STAT_UNCORR
ETHTOOL_A_FEC_STAT_CORR_BITS
Command: ethtool -I show-fec <ifc> |
Application Device Queues | [ConnectX-4 and above] Added driver-level support for Application Device Queues. This feature allows partition defining over the RX/TX queues into groups and isolates traffic of different applications. This mainly improves predictability and tail latency. |
Reinjection of Packets Into Kernel | [All HCAs] Added support for a new software steering action, mlx5dv_dr_action_create_dest_root_table(). This action can be used to forward packets back into a level 0 table. As a table with level 0 is the kernel owned table, this will result in injecting packets to the kernel steering pipeline. |
DCT LAG | [ConnectX-6 Dx and above] Added firmware support to allow explicit port selection based on steering and not QP affinity. Functionality:
|
AES-XTS in RDMA | Added support for plaintext AES-XTS DEKs. |
General | |
Feature/Change | Description |
23.10-1.1.9.0 | |
Lightweight Local SFs | Following the addition of the Lightweight Local SFs feature in version 23.07, in order to configure the scalable-functions, follow the revised instructions as detailed in the Step-by-Step Guide. Note: "Step 2.9 - Set all SF specific device parameters" is now mandatory for local SFs. |
23.10-0.5.5.0 | |
Customer Affecting Change | Description |
Debugfs Directory Path Change | The debugfs directory of each interface can now be found under: /sys/kernel/debug/mlx5//, and not directly under the root of the debugfs filesystem (/sys/debug/kernel). |
Deprecation of OFED Public Power PC Installation | Starting from this release, MLNX_OFED releases for Power PC are no longer available for download from the public Download Center web page. Instead, you can find it on the following page: https://network.nvidia.com/support/firmware/ibm-systemp/. |
Pre-notification: Deprecation of Older Operating Systems | Starting from next release, MLNX_OFED releases will no longer support operating systems with kernels below v4.18. This includes the following systems:
|
Customer Affecting Change | Description |
23.07-0.5.1.2 | |
Creating a QKEY with an MSB Set | To allow non-privileged users to create a QKEY with an MSB set, a new module parameter was added. For details, please see “QKEY Mitigation in the Kernel“ under New Features. |
23.07-0.5.0.0 | |
IRQ Naming | IRQ renaming is no longer done when bringing the interface up/down. The IRQ name is now constant and is not affected by the interface state. |
RPM Packages Verification Key | RPM_GPG-KEY-Mellanox (or its variants) is no longer the public key that verifies RPM packages of MLNX_OFED. Instead, the RPM_GPG-KEY-Mellanox file on the top-level directory of the ISO should be used. |
Hairpin sysfs Support | Hairpin sysfs support was restricted to physical and virtual functions only. |
mlx5_core node_guid Module Parameter | Removed a non-functional mlx5_core node_guid module parameter. |
OpenSM Init | Starting from this release, the opensm init service moves from init.d ( |
Apt Signing Key | Starting from this release, the public key that signed the apt repository of MLNX_OFED is included in the ISO in a format that can be used directly by the apt for repository signatures verification. |
IPoIB ULP Mode Deprecation | Starting from this release, MLNX_OFED supports IPoIB enhanced mode only. The ability to switch back to ULP mode using ipoib_enhanced module parameter is not supported. For more information about the enhanced mode, please refer to the OFED user manual, example: Enhanced IP over InfiniBand. |
Pre-notification: Deprecation of OFED Public Power PC Installation | Starting from next release, MLNX_OFED releases for Power PC will no longer be available for download from the public Download Center web page. |
Customer Affecting Change | Description |
23.07-0.5.0.0 | |
Creating a QKEY with an MSB Set | To allow non-privileged users to create a QKEY with an MSB set, a new module parameter was added. For details, please see “QKEY Mitigation in the Kernel“ under New Features. |
IRQ Naming | IRQ renaming is no longer done when bringing the interface up/down. The IRQ name is now constant and is not affected by the interface state. |
RPM Packages Verification Key | RPM_GPG-KEY-Mellanox (or its variants) is no longer the public key that verifies RPM packages of MLNX_OFED. Instead, the RPM_GPG-KEY-Mellanox file on the top-level directory of the ISO should be used. |
Hairpin sysfs Support | Hairpin sysfs support was restricted to physical and virtual functions only. |
mlx5_core node_guid Module Parameter | Removed a non-functional mlx5_core node_guid module parameter. |
OpenSM Init | Starting from this release, the opensm init service moves from init.d ( |
Apt Signing Key | Starting from this release, the public key that signed the apt repository of MLNX_OFED is included in the ISO in a format that can be used directly by the apt for repository signatures verification. |
IPoIB ULP Mode Deprecation | Starting from this release, MLNX_OFED supports IPoIB enhanced mode only. The ability to switch back to ULP mode using ipoib_enhanced module parameter is not supported. For more information about the enhanced mode, please refer to the OFED user manual, example: Enhanced IP over InfiniBand. |
Pre-notification: Deprecation of OFED Public Power PC Installation | Starting from next release, MLNX_OFED releases for Power PC will no longer be available for download from the public Download Center web page. |
Customer Affecting Change | Description |
23.04-0.5.3.3 | |
Netdev Interface Configuration is not Preserved During Reload/Reset/Recovery | As of OFED 23.04, during reset/reload/recovery flows, the netdev interface is destroyed and re-created (rather than just suspended). As a result, the netdev interface configuration is not preserved, and must be re-applied. The way to do this is to use proper network-scripts and/or udev rules files to configure network interface parameters. These are automatically triggered whenever a netdev interface is added, regardless of whether it was added due to a user-initiated operation or an automatic failure recovery operation. Thus, no special processing is required to re-apply the network interface configuration parameters following a reset/reload/recovery operation – it is performed automatically. |
Prenotification: Deprecation of OFED Public Power PC Installation | Starting next release, MLNX_OFED releases for Power PC will no longer be available for download from the public Download Center web page. |
Prenotification: ULP Mode Deprecation | Starting next release, MLNX_OFED will support IPoIB enhanced mode only. The ability to switch back to ULP mode using ipoib_enhanced module param will not be supported. For more information about the enhanced mode, please refer to OFED user manual, example: Enhanced IP over InfiniBand |
Installation, ISO, RedHat | In order to address RHEL kernel symbol changes, ISO images for the following operating systems are built with the updated kernel versions as follows:
This change comes to support RedHat updated kernels without the need to add --add-kernel-support during OFED installation. |
Power Setups on UCX/HPC-X | UCX/HPC-X no longer supports Power setups. |
NEO-Host | Starting from this release, OFED will discontinue the provision of NEO-Host. NEO-Host can be manually downloaded and installed using the following guide: https://docs.nvidia.com/networking/display/NEOSDKv26/Installation+and+Initial+Configuration#InstallationandInitialConfiguration-DownloadingtheMellanoxNEOSDKSoftware |
dapl | Starting from this release, OFED will discontinue the provision of dapl. |
Signing Key for SLES15 sp4 and sp5 | As of version 23.04, the builds for SLES15 sp4 and sp5 are being singed with a newer signing key. The corresponding public key can be downloaded from https://www.mellanox.com/downloads/ofed/nv_nbu_kernel_signing_key_pub.der instead of https://www.mellanox.com/downloads/ofed/mlnx_signing_key_pub.der. |
dump_pr SM Plugin | Starting from this release, OFED will discontinue the provision of dump_pr subnet manager plugin. |
mpi-selector | Starting from this release, OFED will discontinue the provision of mpi-selector. |
OpenSM Init | Starting 23.07 release, opensm init service will move from init.d to systemd. |
Customer Affecting Change | Description |
5.9-0.5.6.0 | |
Deprecation, LAG Mode via Sysfs | Setting LAG mode via Sysfs is going to be deprecated in a future release. Instead, LAG Hash mode will be used by default, similar to upstream behavior. |
LAG Configuration, PCI Error | From version 5.9, LAG configuration will be lost in case driver incurs a PCI error. Make sure to reconfigure the bond after driver completes the recovery from the PCI error. In releases prior to 5.9, in case of PCI error (EEH injections on PPC setup), the driver recovers LAG bond and reconfigures it automatically in case it what configured before the appearance of the error. |
Customer Affecting Change | Description |
5.7-1.0.2.0 | |
Multi-Block Encryption | Multi-block encryption is currently unsupported, due to a hardware limitation. |
Feature/Change | Description |
5.6-2.0.9.0 | |
Operating Systems | Added support for the following Operating Systems: RHEL8.6, RHEL9.0, SLES15-SP4. |
General | |