Changes and New Features

Adapters

Added support for ConnectX-6 Dx adapter cards.

Userspace Software Steering ConnectX-6 Dx Support

[Beta] Added support for software steering on ConnectX-6 Dx adapter cards in the user-space RDMA-Core library through the mlx5dv_dr API.

Virtual Output Queuing (VoQ) Counters

Exposed rx_prio[p]_buf_discard, rx_prio[p]_wred_discard and rx_prio[p]_marked firmware counters that count the number of packets that were dropped due to insufficient resources.

IPsec Crypto Offloads

[Beta] IPsec crypto offloads are now supported on ConnectX-6 Dx devices and up. The offload functions use the existing ip xfrm tool to activate offloads on the device. It supports transport/tunnel mode with AES-GCM IPsec scheme.

TLS TX Hardware Offload

[Alpha] Added support for hardware offload encryption of TLS traffic.

VirtIO Acceleration through Datapath I/O Processor (vDPA)

Added support to enable mapping the VirtIO access region (VAR) to be used for doorbells by vDPA applications. Specifically, the following DV APIs were introduced (see man page for more details):

• mlx5dv_alloc_var()

• mlx5dv_free_var()

Resource Allocation on External Memory

Added support to enable overriding mlx5 internal allocations in order to let applications allocate some resources on external memory, such as that of the GPU.

The above is achieved by extending the parent domain object with custom allocation callbacks. Currently supported verbs objects are: QP, DBR, RWQ, SRQ.

Hardware Clock Exposure

Added support for querying the adapter clock via mlx5dv_query_device.

ODP Diagnostic Counters

Added ODP diagnostics counters for the following items per MR (memory region) within IB/mlx5 driver:

1. Page faults: Total number of faulted pages.

2. Page invalidations: Total number of pages invalidated by the OS during all invalidation events. The translations can no longer be valid due to either non-present pages or mapping changes.

3. Prefetched pages: When prefetching a page, a page fault is generated in order to bring the page to the main memory.

Devlink Health CR-Space Dump

Added the option to dump configuration space via the devlink tool in order to improve debug capabilities.

Multi-packet TX WQE Support for XDP Transmit Flows

The conventional TX descriptor (WQE or Work Queue Element) describes a single packet for transmission. Added driver support for the HW feature of multi-packet TX WQEs in XDP transmit flows. With this, the HW becomes capable of working with a new and improved WQE layout that describes several packets. In effect, this feature saves PCI bandwidth and transactions, and improves transmit packet rate.

OVS-Kernel ToS Rewrite

Added support for Type of Service (ToS) rewrite in the OVS-Kernel.

OVS-Kernel Mirroring

Added support for mirroring output in SwitchDev mode in the OVS-Kernel. The mirroring port may either be a local or a remote VF, using VxLAN or GRE encapsulations.

GENEVE Encap/Decap Rules Offload

Added support for GENEVE encapsulation/decapsulation rules offload.

GPRS Tunneling Protocol (GTP) Header

[Beta] Added support for matching (filtering) GTP header-based packets using mlx5dv_dr API over user-space RDMA-Core library.

Multi Packet Tx WQE Support for XDP Transmit Flows

Added driver support for the hardware feature of multi-packet Tx to work with a new and improved WQE layout that describes several packets instead of a single packet for XDP transmission flows. This saves PCI bandwidth and transactions, and improves transmit packet rate.

Userspace Software Steering Debugging API

[Beta] Added support for software steering to dump flows for debugging purposes in the user-space RDMA-Core library through the mlx5dv_dr API.

Kernel Software Steering for Connection Tracking (CT)

[Beta] Added support for updating CT rules using the software steering mechanism.

Kernel Software Steering Remote Mirroring

[Beta] Added support for updating remote mirroring rules using the software steering mechanism.

OVS-DPDK Support

Added OVS-DPDK component as part of the MLNX_OFED package with hardware offload capabilities.

OVS-DPDK Connection Tracking

[Beta] Added support for OvS-DPDK Connection Tracking hardware offload.

OVS-DPDK VirtIO Acceleration through VF Relay

Added support for OVS-DPDK VirtIO Acceleration through VF Relay (also known as Software vDPA) forwarding of traffic from VF to Virtio VM and vice-versa.

OVS-DPDK VXLAN Encap/Decap

Added support for OVS-DPDK VXLAN encapsulation and decapsulation hardware offload.

Discard Counters

Exposed rx_prio[p]_discards discard counters per priority that count the number of received packets dropped due to lack of buffers on the physical port.

MPLS Traffic

Added support for reporting TSO and CSUM offload capabilities for MPLS tagged traffic and, allowed the kernel stack to use these offloads.

mlx5e Max Combined Channels

Increased the driver’s maximal combined channels value from 64 to 128 (however, note that OOB value will not cross 64).

128 is the upper bound. Lower maximal value can be seen on the host, depending on the number of cores and MSIX's configured by the firmware.

RoCE Accelerator Counters

Added the following RoCE accelerator counters:

• roce_adp_retrans - counts the number of adaptive retransmissions for RoCE traffic

• roce_adp_retrans_to - counts the number of times RoCE traffic reached timeout due to adaptive retransmission

• roce_slow_restart - counts the number of times RoCE slow restart was used

• roce_slow_restart_cnps - counts the number of times RoCE slow restart generated CNP packets

• roce_slow_restart_trans - counts the number of times RoCE slow restart changed state to slow restart

Migration to RDMA-Core

The default installation of the userspace is now the RDMA-Core library instead of the legacy verbs. This achieves most of the legacy experimental verbs’ functionalities, and more.

For Mellanox VMA or Mellanox RiverMax, use experimental verbs (prefix “ibv_exp”).

For further information on the migration to RDMA-Core and the list of new APIs used for various MLNX_OFED features, please refer to the Migration to RDMA-Core document.

ibdev2netdev Tool Output

ibdev2netdev tool output was changed such that the bonding device now points at the bond instead of the slave interface.

Memory Region

Added support for the user to register memory regions with a relaxed ordering access flag. This can enhance performance, depending on architecture and scenario.

Devlink Health Reporters

Added support for monitoring and recovering from errors that occur on the RX queue, such as cookie errors and timeout.

GSO Optimization

Improved GSO (Generic Segmentation Offload) workload performance by decreasing doorbells usage to the minimum required.

TX CQE Compression

Added support for TX CQE (Completion Queue Element) compression. Saves on outgoing PCIe bandwidth by compressing CQEs together. Disabled by default. Configurable via private flags of ethtool.

Firmware Versions Query via Devlink

Added the option to query for running and stored firmware versions using the devlink tool.

Firmware Flash Update via Devlink

Added the option to update the firmware image in the flash using the devlink tool.

Usage: devlink dev flash <dev> file <file_name>.mfa2

For further information on how to perform this update, see "Updating Firmware Using ethtool/devlink and .mfa2 File" section in MFT User Manual.

Devlink Health WQE Dump

Added support for WQE (Work Queue Element) dump, triggered by an error on Rx/Tx reporters. In addition, some dumps (not triggered by an error) can be retrieved by the user via devlink health reporters.

GENEVE Tunnel Stateless Offload

Added support for GENEVE tunneled hardware offloads of TSO, CSUM and RSS.

TCP Segmentation and Checksum Offload

Added TCP segmentation and checksum offload support for MPLS-tagged traffic.

Bug Fixes

See “Bug Fixes" section.