image image image image image

On This Page

New Features

The following are the new features and changes that were added in this version.



ASAP2 Features
Bridge Offloads with VLAN

[ConnectX-4 and above] Added support for bridge offloads with VLAN support that works on top of mlx5 representors in switchdev mode. 

Supporting OVS Groups in Fast-Failover Mode

[ConnectX-6 Dx] Improved OVS failover through support for OVS groups in fast-failover mode + VF_LAG configuration with OVS.

Exposing Hairpin Queues Information

[ConnectX-6 Dx and BlueField-2] Added support for exposing hairpin out of buffer drop counter per device.  This feature shows buffer drops related only to hairpin queues which were opened on the queried device.

To enable this counting mode (this must be done before any hairpin rules are created), use the following: echo "on <peer_devname>" > /sys/class/net/<dev>/hp_oob_cnt_mode where <peer_devname> is the peer device to which traffic coming to the configured device will be forwarded to for transmission.

To read the drop counter, use the following: cat /sys/class/net/<dev>/hp_oob_cnt

Linux Bridge Offload

[ConnectX-6 Dx and BlueField-2] Added bridge offloads to support bonding (VF LAG), attaching bond device to bridge instead of uplink representors. 

VLAN Pop/Push

[ConnectX-6 Dx] Added OOB support for VLAN push on Rx (wire to VF) and VLAN pop on Tx (wire to VF) in switchdev mode. 

Offload Forwarding to Multiple Destinations
[ConnectX-5 and above] Added support for offloading packet replication to up to 32 destination through the use of TC rule.

Slow Path Metering

[ConnectX-4 and above] Expanding the RDMA statistic tool to support setting vendor-specific optional counters dynamically using netlink.
Added to mlx5_ib the following optional counters:

$ rdma statistic mode supported link rocep8s0f0/1
link rocep8s0f0/1 supported optional-counters cc_rx_ce_pkts,cc_rx_cnp_pkts,cc_tx_cnp_pkts

$ sudo rdma statistic set link rocep8s0f0/1 optional-counters cc_rx_ce_pkts,cc_rx_cnp_pkts

$ rdma statistic mode link rocep8s0f0/1
link rocep8s0f0/1 optional-counters cc_rx_ce_pkts,cc_rx_cnp_pkts

$ sudo rdma statistic set link rocep8s0f0/1 optional-counters cc_rx_ce_pkts

$ rdma statistic mode link rocep8s0f0/1
link rocep8s0f0/1 optional-counters cc_rx_ce_pkts

$ sudo rdma statistic unset link rocep8s0f0/1 optional-counters

Core Features
Subfunction Trust Configuration Enhancement

[ConnectX-5 and above] Added support via mlxdevm to mark a given PCI subfunction (SF) or virtual function (VF) as a trusted function. The device/firmware decides how to define privileges and access to resources. 

Prevent VF Memory Exhaustion

[All] Added support for preventing VF memory exhaustion. This feature exposes a sysfs (to the system admin) which can set a limit on each VF memory consumption.

Note: Currently only supported on Ethernet. 

BlueField NIC Separate Reset

[BlueField-2] Added support for resetting the NIC domain of BlueField-2 while keeping ARM alive. 

Multiple Steering Priorities for FDB Rules

[ConnectX-6 Dx and BlueField-2] Added support in multiple flow steering priorities for FDB rules. 

NetDev Features
Traffic Engineering: Hierarchical QoS

[ConnectX-5 and above] Added support for offloading the HTB qdisc to the NIC, allowing it to scale better by eliminating a single locking point. The configuration is done with the TC commands.

Note: Kernel 5.15 or higher is required. Limited to 256 nodes.

TLS RX Resynchronization Resiliency Feature Description

[ConnectX-6 Dx and above] Added support for driver resiliency against high load of RX resync operations. 

Simultaneous PTP and CQE Compression

Added support for the activation of PTP and CQE compression simultaneously. Since CQE compression might harm the accuracy of the PTP, the feature enables PTP packets to be moved to a dedicated queue where they are not subjected to compression. However, this configuration conflicts with setting aRFS. Turning off CQE compression, causes a hiccup in traffic which may cause a loss of synchronization. To overcome this, restart the synchronization.

Note: This combination is supported only for Ethernet drivers. Other driver profiles, like IPoIB and representors, do not support this combination. 

Installation Features
Multiple Development Headers Packages 

Allowed installing multiple mlnx-ofa_kernel development headers packages (for different kernel versions of the same mlnx-ofa_kernel package version) side by side on the same system. 

Kernel Module Signature

Added signature of kernel modules of EulerOS 2.0 SP8-SP10 (x86_64 and aarch64) builds of MLNX_OFED. 

Enable sf-cfg-drv by Default in EulerOS2.0

Enabled SF_CFG (SF config dummy driver, --with-sf-cfg-drv) on EulerOS2.0 SP8 and SP10. 

For additional information on the new features, please refer to EN_OFED User Manual

Customer Affecting Changes

Customer Affection ChangeDescription


Disabling RoCE While Using sysfs

When using sysfs to enable/disable roce in kernel 5.5 and up,  the "devlink reload" command (using iproute2 with devlink tool) will need to be used to activate the RoCE status change.

Disable RoCE example:
1. echo 0 > /sys/bus/pci/devices/0000:08:00.0/roce_enable
2. devlink dev reload pci/0000:08:00.0 

mlnx-ofa_kernel Installation

The source code for mlnx-ofa_kernel is no longer installed by default on RPM-based distributions (e.g., RHEL and SLES). 

mlnx-ofa_kernel is included in the <<package mlnx-ofa_kernel-devel-source>> in the MLNX_OFED distributions under RPMS/ and may be manually installed from there.
• There is no change for deb-based distributions (Debian and Ubuntu). The full source is included, as before, in the package mlnx-ofed-kernel-dkms.

Software Encapsulation Compatibility

There is an encapL2 compatibility issue with accelerated reformat action creation using mlx5dv_dr API.
Using OFED 5.4 with firmware xx.32.1xxx and above or using OFED 5.5 with firmware lower than xx.32.1xxx will not allow accelerated reformat action. (Using OFED 5.4 and 5.5 with bundle firmware works properly.)

xpmem in RHEL8
Added xpmem packages in RHEL8 builds.


Starting OVS DPDK 2.15, the Python minimum required version is 3 and OVS-DPDK will not be compiled using Python 2.