Release Notes Change Log History

VFs per PF

Increased the amount of maximum virtual functions (VF) that can be allocated to a physical function (PF) to 127 VF.

SW-Defined UDP Source Port for RoCE v2

UDP source port for RoCE v2 packets is now calculated by the driver rather than the firmware, achieving better distribution and less congestion. This mechanism works for RDMA- CM QPs only, and ensures that RDMA connection messages and data messages have the same UDP source port value.

Local Loopback Disable

Added the ability to manually disable Local Loopback regardless of the number of open user-space transport domains.

Adapter Cards

Added support for ConnectX-6 Ready. For further information, please contact Mellanox Support.

NEO-Host

Integrated NEO-Host for orchestration and management of host networking into MLNX_OFED package.

Bug Fixes

See Bug Fixes section.

Bug Fixes

See Bug Fixes section.

Adaptive Interrupt Moderation

Added support for adaptive Tx, which optimizes the moderation values of the Tx CQs on runtime for maximum throughput with minimum CPU overhead.

This mode is enabled by default.

Updated Adaptive Rx to ignore ACK packets so that queues that only handle ACK packets remain with the default moderation.

Docker Containers [Beta]

Added support for Docker containers to run over Virtual RoCE and InfiniBand devices using SR-IOV mode.

VF Statistics

Performed the following virtual function statistics changes:

• Added tx_broadcast and tx_multicast counters

• Included RDMA statistics for existing counters

Force TTL

Added support for setting a global TTL value for all RC QPs and rdma-cm QPs.

Firmware Tracer

Added a new mechanism for the device’s FW/HW to log important events into the event tracing system (/sys/kernel/debug/tracing) without requiring any Mellanox-specific tool.

Note: This feature is enabled by default.

CR-Dump

Accelerated the original cr-dump by optimizing the reading process of the device’s CR-Space snapshot.

RoCE ICRC Error Counter

Added support for a new counter that exposes the amount of corrupted RoCE packets that arrive with bad Invariant Cyclic Redundancy Code (ICRC).

VST Q-in-Q

Added support for C-tag (0x8100) VLAN insertion to tagged packets in VST mode.

Ethernet Tunneling Over IPoIB Driver (eIPoIB)

Re-added support for eth_ipoib driver, which provides a standard Ethernet interface to be used as a Physical Interface (PIF) into the Hypervisor virtual network, and serves one or more Virtual Interfaces (VIF).

OVS Offload using ASAP2

Added support for Mellanox Accelerated Switching And Packet Processing (ASAP2) technology, which allows OVS offloading by handling OVS data-plane, while maintaining OVS control-plane unmodified. OVS Offload using ASAP2 technology provides significantly higher OVS performance without the associated CPU load.

For further information, refer to ASAP2 Release Notes under www.mellanox.com -> Products -> Software -> ASAP2

Upstream Libraries

Added a repository repodata to support installing upstream libraries (based on upstream rdma-core), using he Operating System's standard package manager (yum, apt-get, etc.).

For further information, please refer to “Installing Upstream rdma-core Libraries” section in MLNX_OFED User Manual

Note: This is intended only for DPDK users.

Installation

Added support for new metadata packages that only install userspace packages at a time (without any kernel packages), using the Operating System's standard package manager (yum, apt-get, etc.). These metadata packages will have the suffix “-user-only”. For example: “mlnx-ofed-all-user-only”.

Bug Fixes

See Bug Fixes section.

Adaptive Interrupt Moderation

Added support for adaptive Tx, which optimizes the moderation values of the Tx CQs on runtime for maximum throughput with minimum CPU overhead.

This mode is enabled by default.

Updated Adaptive Rx to ignore ACK packets so that queues that only handle ACK packets remain with the default moderation.

Docker Containers [Beta]

Added support for Docker containers to run over Virtual RoCE and InfiniBand devices using SR-IOV mode.

VF Statistics

Performed the following virtual function statistics changes:

• Added tx_broadcast and tx_multicast counters

• Included RDMA statistics for existing counters

Force TTL

Added support for setting a global TTL value for all RC QPs and rdma-cm QPs.

Firmware Tracer

Added a new mechanism for the device’s FW/HW to log important events into the event tracing system (/sys/kernel/debug/tracing) without requiring any Mellanox-specific tool.

Note: This feature is enabled by default.

CR-Dump

Accelerated the original cr-dump by optimizing the reading process of the device’s CR-Space snapshot.

RoCE ICRC Error Counter

Added support for a new counter that exposes the amount of corrupted RoCE packets that arrive with bad Invariant Cyclic Redundancy Code (ICRC).

VST Q-in-Q

Added support for C-tag (0x8100) VLAN insertion to tagged packets in VST mode.

Ethernet Tunneling Over IPoIB Driver (eIPoIB)

Re-added support for eth_ipoib driver, which provides a standard Ethernet interface to be used as a Physical Interface (PIF) into the Hypervisor virtual network, and serves one or more Virtual Interfaces (VIF).

OVS Offload using ASAP2

Added support for Mellanox Accelerated Switching And Packet Processing (ASAP2) technology, which allows OVS offloading by handling OVS data-plane, while maintaining OVS control-plane unmodified. OVS Offload using ASAP2 technology provides significantly higher OVS performance without the associated CPU load.

For further information, refer to ASAP² Release Notes under www.mellanox.com -> Products -> Software -> ASAP2

Upstream Libraries

Added a repository repodata to support installing upstream libraries (based on upstream rdma-core), using the Operating System's standard package manager (yum, apt-get, etc.).

For further information, please refer to “Installing Upstream rdma-core Libraries” section in MLNX_OFED User Manual

Note: This is intended only for DPDK users.

Installation

Added support for new metadata packages that only install userspace packages at a time (without any kernel packages), using the Operating System's standard package manager (yum, apt-get, etc.). These metadata packages will have the suffix “-user-only”. For example: “mlnx-ofed-all-user-only”.

Bug Fixes

See Bug Fixes section.

Multi-Packet Work Request (WR)

Added support for the following multi-packet WR related verbs for control path:

• ibv_exp_query_device

• ibv_exp_create_srq

For further information on the use of these verbs, please refer to the Verbs man page.

Coherent Accelerator Processor Interface (CAPI) [beta]

Added support for CAPI, an interface that enables

ConnectX-5 adapter cards to provide the best performance for Power and OpenPower based platforms.

Tunneled Atomic

Added support for RDMA atomic commands offload so that when an RDMA Write operation is issued, the payload indicates which atomic operation to perform, instead of being written to the Memory Region (MR).

Packet Pacing

Added support for the following advanced burst control parameters:

• max_burst_sz - for indicating the maximal burst size of packets

• typical_pkt_sz - for improving the accuracy of the rate limiter

Erasure Coding

Offload verbs

Added support for erasure coding offload software verbs (encode/decode/update API) supporting a number of redundancy blocks (m) greater than 4.

Virtual MAC

Removed support for Virtual MAC feature.

RoCE LAG

Added out of box RoCE LAG support for RHEL 7.2 and RHEL 6.9.

Dropped Counters

Added a new counter rx_steer_missed_packets which provides the number of packets that were received by the NIC, yet were discarded/dropped since they did not match any flow in the NIC steering flow table.

Added the ability for SR-IOV counter rx_dropped to count the number of packets that were dropped while vport was down.

Relaxed Ordering (RSYNC)

Added support for RSYNC feature to ensure correct ordering of memory operations between the GPU and HCA.

Reset Flow

Added support for triggering software reset for firmware/driver recovery. When fatal errors occur, firmware can be reset and driver reloaded.

Striding RQ with HW Time-Stamping

Added the option to retrieve the HW timestamp when polling for completions from a completion queue that is attached to a multi-packet RQ (Striding RQ).

DSCP Trust Mode

Added support for automatically setting the number of TC to 8 when the Trust state is changed to DSCP.

Receive Buffer

Added xon and xoff columns to the Receive Buffer configuration display.

Physical Address Memory Allocation

Added support to register a specific physical address range.

Innova IPsec Adapter Cards

Added support for Mellanox Innova IPsec EN adapter card, that provides security acceleration for IPsec-enabled networks.

Precision Time Protocol (PTP)

Added support for PTP feature over PKEY interfaces.

This feature allows for accurate synchronization between the distributed entities over the network. The synchronization is based on symmetric Round Trip Time (RTT) between the master and slave devices, and is enabled by default.

1PPS Time Synchronization

Added support for One Pulse Per Second (1PPS) over IPoIB interfaces.

Virtual MAC

Added support for Virtual MAC feature, which allows users to add up to 4 virtual MACs (VMACs) per VF. All traffic that is destined to the VMAC will be forwarded to the relevant VF instead of PF. All traffic going out from the VF with source MAC equal to VMAC will go to the wire also when Spoof Check is enabled.

For further information, please refer to “Virtual MAC” section in MLNX_OFED User Manual.

Receive Buffer

Added the option to change receive buffer size and cable length. Changing cable length will adjust the receive buffer's xon and xoff thresholds.

For further information, please refer to “Receive Buffer” section in MLNX_OFED User Manual.

GRE Tunnel Offloads

Added support for the following GRE tunnel offloads:

• TSO over GRE tunnels

• Checksum offloads over GRE tunnels

• RSS spread for GRE packets

NVMEoF

Added support for the host side (RDMA initiator) in RedHat 7.2 and above.

Dropless Receive Queue (RQ)

Added support for the driver to notify the FW when SW receive queues are overloaded.

PFC Storm Prevention

Added support for configuring PFC stall prevention in cases where the device unexpectedly becomes unresponsive for a long period of time. PFC stall prevention disables flow control mechanisms when the device is stalled for a period longer than the default pre-configured timeout. Users now have the ability to change the default timeout by moving to auto mode.

For further information, please refer to “PFC Stall Prevention” section in MLNX_OFEDUser Manual.

Force DSCP

Added support for this feature that enables setting a

global traffic_class value for all RC QPs.

Q-in-Q

Added support for Q-in-Q VST feature in ConnectX-5 adapter cards family.

Device Memory Programming [beta]

Added support for on-chip memory allocation and usage in send/receive and RDMA operations at beta level.

Virtual Guest Tagging (VGT+)

Added support for VGT+ in ConnectX-4/ConnectX-5 HCAs. This feature is s an advanced mode of Virtual Guest Tagging (VGT), in which a VF is allowed to tag its own packets as in VGT, but is still subject to an administrative VLAN trunk policy. The policy determines which VLAN IDs are allowed to be transmitted or received. The policy does not determine the user priority, which is left unchanged.

For further information, please refer to “Virtual Guest Tagging (VGT+)” section in MLNX_OFED User Manual.

Tag Matching Offload

Added support for hardware Tag Matching offload with Dynamically Connected Transport (DCT).

Shared Memory Region (MR)

Removed support for Shared MR feature on ConnectX-3/ConnectX-3 Pro adapter cards. As a result of this change, the following API/flags should not be used:

• ibv_exp_reg_shared_mr

• access shared flags for ibv_exp_reg_mr (IBV_EXP_ACCESS_SHARED_MR_XXX)

CR-DUMP

Added support for the driver to take an automatic snapshot of the device’s CR-Space in cases of critical failures.

For further information, please refer to “CRDUMP” section in MLNX_OFED User Manual.

Upstream Libraries

Added the option to install upstream libraries (based on upstream rdma-core) for DPDK users only.

For further information, please refer to “Installing Upstream rdma-core Libraries” section in MLNX_OFED User Manual.

DiSNI

Added the option to install libdisni package as part of MLNX_OFED.

For further information, please refer to section “Installing libdisni Package” in MLNX_OFED User Manual.

Service Scripts

Added the ability to disable the ‘ stop ’ option in the openibd service script, by setting ALLOW_STOP=no in

/etc/infiniband/openib.conf.

Starting from the next release, ‘ stop ’ option will be disabled by default, and in order to enable it, ALLOW_STOP should be set to ‘yes’ in the conf file, or force-stop should be run.

RoCE Diagnostics and ECN Counters

Added support for additional RoCE diagnostics and ECN congestion counters under /sys/class/infiniband/mlx5_0/ports/1/hw_counters/ directory.

For further information, refer to the Understanding mlx5 Linux Counters and Status Parameters Community post.

rx-fcs Offload (ethtool)

Added support for rx-fcs ethtool offload configuration. Normally, the FCS of the packet will be truncated by the ASIC hardware before sending it to the application socket buffer (skb). Ethtool allows to set the rx-fcs not to be truncated, but to pass it to the application for analysis.

For more information and usage, refer to Understanding ethtool rx-fcs for mlx5 Drivers Community post.

DSCP Trust Mode

Added the option to enable PFC based on the DSCP value. Using this solution, VLAN headers will no longer be mandatory for use.

For further information, refer to the HowTo Configure Trust Mode on Mellanox Adapters Community post.

RoCE ECN Parameters

ECN parameters have been moved to the following directory: /sys/kernel/debug/mlx5/<PCI BUS>/cc_params/

For more information, refer to the HowTo Configure DCQCN (RoCE CC) for ConnectX-4 (Linux) Community post.

Flow Steering Dump Tool

Added support for mlx_fs_dump, which is a python tool that prints the steering rules in a readable manner.

Secure Firmware Updates

Firmware binaries embedded in MLNX_OFED package now support Secure Firmware Updates. This feature provides devices with the ability to verify digital signatures of new firmware binaries, in order to ensure that only officially approved versions are installed on the devices.

For further information on this feature, refer to Mellanox Firmware Tools (MFT) User Manual.

Enhanced IPoIB

Added support for Enhanced IPoIB feature, which enables better utilization of features supported in ConnectX-4 adapter cards, by optimizing IPoIB data path and thus, reaching peak performance in both bandwidth and latency.

Enhanced IPoIB is enabled by default.

PeerDirect

Added the ability to open a device and create a context while giving PCI peer attributes such as name and ID.

For further details, refer to the PeerDirect Programming Community post.

Probed VFs

Added the ability to disable probed VFs on the hypervisor. For further information, see HowTo Configure and Probe VFs on mlx5 Drivers Community post.

Local Loopback

Improved performance by rendering Local loopback (unicast and multicast) disabled by mlx5 driver by default while local loopback is not in use. The mlx5 driver keeps track of the number of transport domains that are opened by user-space applications. If there is more than one user-space transport domain open, local loopback will automatically be enabled.

1PPS Time Synchronization (at alpha level)

Added support for One Pulse Per Second (1PPS), which is a time synchronization feature that allows the adapter to send or receive 1 pulse per second on a dedicated pin on the adapter card.

For further information on this feature, refer to the HowTo Test 1PPS on Mellanox Adapters Community post.

Precision Time Protocol (PTP)

Added support for PTP feature in IPoIB offloaded devices.

This feature allows for accurate synchronization between the distributed entities over the network.

The synchronization is based on symmetric Round Trip Time (RTT) between the master and slave devices.

The feature is enabled by default.

For further information, refer to Running Linux PTP with ConnectX-4 Community post.

Fast Driver Unload

Added support for fast driver teardown in shutdown and kexec flows.

NVMEoF Target Offload

Added support for NVMe over fabrics (NVMEoF) offload, an implementation of the new NVMEoF standard target (server) side in hardware.

For further information on NVMEoF Target Offload, refer to HowTo Configure NVMEoF Target Offload .

MPI Tag Matching

Added support for offloading MPI tag matching to HCA.

RDMA CM

Changed the default RoCE mode on which RDMA CM runs to RoCEv2 instead of RoCEv1.

RDMA_CM session requires both the client and server sides to support the same RoCE mode. Otherwise, the client will fail to connect to the server.

For further information, refer to RDMA CM and RoCE Version Defaults Community post.

Lustre

Added support for Lustre file system open-source project.

Operating Systems

Added support for Ubuntu v17.04.

PCIe Error Counting

[ConnectX-4/ConnectX-4 Lx] Added the ability to expose physical layer statistical counters to ethtool.

Multiprotocol Label Switching (MPLS) Tagged Packets Classification

[ConnectX-4/ConnectX-4 Lx] Enabled packet flow steering rules with IPv4/IPv6 classification (for raw packet QP (DPDK) only) to work on IPv4/IPv6 over MPLS (Ethertype 0x8847 and 0x8848) encapsulated packets.

RoCE VFs

[ConnectX-4/ConnectX-4 Lx] Added the ability to enable/disable RoCE on VFs.

RoCE LAG

[ConnectX-4/ConnectX-4 Lx] Added support for RoCE over LAG interface.

Standard ethtool

[ConnectX-4/ConnectX-4 Lx] Added support for flow steering and rx-all mode.

SR-IOV Bandwidth Share for Ethernet/RoCE (beta)

[ConnectX-4/ConnectX-4 Lx] Added the ability to guarantee the minimum rate of a certain VF in SR-IOV mode.

Adapter Cards

Added support for ConnectX-5 and ConnectX-5 Ex HCAs.

DSCP ConfigFS Control for RDMA-CM QPs

Added the ability to configure ToS/DSCP for RDMA-CM QPs only.

Soft RoCE (beta)

Add software implementation of RoCE that allows RoCE to run on any Ethernet network adapter whether it offers hardware acceleration or not.

NVMe over Fabrics (NVMEoF)

NVMEoF related module installation has been disabled by default. In order to enable it, add the “ --with-nvmf ” installation option to the “mlnxofedinstall” script.

NFS over RDMA (NFSoRDMA)

Removed support for NFSoRDMA drivers. These drivers are no longer provided along with the MLNX_OFED package.