Changes and New Features

Component

Feature/Change

Description

mlx5

Patterns and Arguments Support

[ConnectX-6 Dx and above] Added a new design of modify_header firmware object. The new approach comprises of two types of objects: Pattern and Argument. Pattern holds header modification templates, later used with corresponding argument object to create complete header modification actions. Single pattern can be used with different arguments in different flows, enabling offload of large number of modify_header flows.

Live Migration

[ConnectX-6 Dx and above] Added a feature that enables migration of a QEMU VM with an assigned VF from one source host to another destination host. This is done as part of the general QEMU migration flow, by suspending the VF on the source, sending all its data to the destination and then resuming the VF on the destination.

[ConnectX-6 Dx and above] Added support for PRE_COPY migration to reduce the downtime of the VM. The optional PRE_COPY state, opens the saving data transfer FD before reaching STOP_COPY, and enables the device to dirty track the internal state changes with the general idea to reduce the volume of data transferred in the STOP_COPY stage.

[ConnectX-6 Dx and above] Added support for Migratable Bit in Live Migration. Since some features cannot be migrated, such as IPsec, for example, when VF is marked as migratable, those features are disabled. Using this feature the user can configure whether VF can be migrated.

[ConnectX-7] Added dirty pages tracking support which enables reducing downtime upon live migration. Once it is used, only the pages that were really dirtied by the device will be marked in QEMU as dirty and will be sent to the target upon stop. Without dirty tracking, all RAM is marked dirty so all RAM is resent upon stop and the downtime is increased.

Support for VM Driven IPsec Full Offload

[ConnectX-7 and above] Added support for IPsec protocol that works with RoCE V2, even when using SR-IOV VFs. The user can configure IPsec while all of the IPsec operation are fully offloaded to the HW, other than the obvious performance improvement. This also enables using IPsec over RoCE packets (which aren't part of the stack), which isn't possible without the full HW offload.

Readiness for CX-7 ETH

[ConnectX-7 and above] Added driver support to enable offloading MACsec. This feature adds support for hardware offloading for MACsec protocol with encryption. Adding, removing and updating SAs/SecYs is supported.

IPsec Full Offload Transport Mode

Added support for IPsec packet offload, an improved version of IPsec crypto mode. In packet offload mode, the hardware is responsible to trim/add headers and to decrypt/encrypt. In this mode, the packet arrives to the stack already decrypted, and vice versa, for TX (exits to hardware as not-encrypted).

xsk Performance Improvements

Added the option to use default RSS functionality to spread traffic across different XSK queues instead of having to provide explicit steering rules.

CT UDP Unidirectional Traffic Offload

[ConnectX-4 and above] Added support for offloaded long-running unidirectional UDP connection with conntrack.

Multi Port E-switch Support

[ConnectX-6 Dx] Enabled multiport of E-Switch A mode, where a single E-Switch is used and all the vports and physical ports on the NIC are connected to it. For example, sending traffic from a VF that is created on PF0 to an uplink that is natively associated with PF1 uplink.

rdma pkts and Bytes C

ounters on VFs Expose to Host

[ConnectX-5 and above] Added RDMA traffic-only counters for rep devices. These counters can now be read from host with ethtool or from sysfs and not only from the container.

Support 256 Bit Keys with kTLS Offload

Added support of kTLS offload with key size of 256 bits.

General Driver Update

Driver base Upstream Kernel v6.3

rdma-core

General Driver Update

Updated to version rdma-core-46.0-1.el9

mstflint

General Driver Update

Updated to version mstflint-4.24.0-1.el9

VMA

General Driver Update

Updated to version libvma-9.8.20-3.el9

UCX

General Driver Update

Updated to version ucx-1.14.1-1.el9.1

© Copyright 2023, NVIDIA. Last updated on Dec 18, 2023.