Changes and New Feature History

PCIe TLP Processing Hints (TPH) and Steering Tag (ST)

Enabled PCIe TLP Processing Hints (TPH) and Steering Tag (ST) during MKey creation.

Note: The steering tag index in the MKey creation must reference an MSIX entry containing the actual steering tag value.

PCIe Congestion Events

Added support for the general PCIe congestion object to monitor and receive events related to inbound and outbound PCIe congestion. A threshold can be configured to specify when the firmware should send an event to the software.

This capability is activated by setting the mlxconfig parameter PCIE_CONGESTION_MONITOR.

RDMA QP

When an RDMA QP encounters a memory access an issue caused by address translation, it can recover without transitioning to an error state. The QP will send an error CQE to notify the software while continuing to serve other VMs and functions.

PPCNT Counters

Firmware now supports new counters in the PPCNT register to track multicast and unicast packets transmitted and received. The counters include:

• port_multicast_xmit_pkts_high

• port_multicast_xmit_pkts_low

• port_multicast_rcv_pkts_high

• port_multicast_rcv_pkts_low

• port_unicast_xmit_pkts_high

• port_unicast_xmit_pkts_low

• port_unicast_rcv_pkts_high

• port_unicast_rcv_pkts_low

Safely Identify DPUs/SmartNICs is a Machine and PCIe Slot

A new access register is introduced that accepts a type, length, and R/W command.

• Write operation: Allocates a new ICMC buffer of the specified size (aligned to 64B) and stores the provided data. If a buffer for the given type already exists, the data in the ICMC is overwritten, and the locked area is adjusted accordingly

• Read operation: If a buffer exists, its data is copied out. If not, the access register returns a size of 0 or an explicit error

The length can be stored within the data in the ICMC, and the type is mapped to 256B chunks (due to access register limitations), so the VA of the buffer is calculated as (base + (type << 8)). The first 4 bytes store a validity flag and the length. If length storage is unnecessary (e.g., null-terminated data), a hardware read can use a cache-line hit as a validity bit.

This feature is designed for limited use cases and does not address multi-host scenarios or broader ICMC utilization implications.

Latency Histogram Counter

Introduced a new latency histogram counter that measures the distribution of read operation latencies from our device to the PCI link, providing better visibility into PCI read performance and potential bottlenecks.

Incoming NC-SI Messages Validation for the payload_len Field

Added an extra validation for the payload_len field in incoming NC-SI messages. Previously, invalid packets might have been accepted; now, such packets are silently dropped.

RSS with Crypto Offload

Added support for RSS with crypto offload enabling the NIC to parallelize packet processing across CPU cores while performing encryption/decryption in hardware. Additionally, introduced a new l4_type_ext parameter with values: 0 (None), 1 (TCP), 2 (UDP), 3 (ICMP).

SPDM

Updated SPDM measurements report to version 1.1.

Bug Fixes

See Bug Fixes in this Firmware Version section.

Bug Fixes

See Bug Fixes in this Firmware Version section.

Static Split 8x100G ConnectX-8 to Spectrum-4 with SM Modules

A static split of 8x100G channels from a ConnectX-8 SuperNIC to a Spectrum-4 switch allows the system to use Single Mode (SM) optical modules for high-speed data transmission across a long-distance fibber link. This setup is typically used in high-performance networks where there is a need for high throughput (e.g., 800G in total bandwidth) with low latency, such as in data centers or high-performance computing environments.

DOCA Telemetry

DOCA Telemetry enables users to monitor and collect data related to the performance, health, and behavior of systems or applications running on DOCA. To optimize for a faster sampling period, it is recommended to configure all PCIe-related Diagnostic Data IDs sequentially, one after another to prevent a prolonged sampling period.

PCIe Switch fwreset

Added support for a new synchronized flow, including a tool and driver, to perform a fwreset on setups with a PCIe switch configuration.

PTP

Unified PTP is now supported across different VFs on the same PF.

Dual-Mode Temperature Compensated Crystal Oscillator (DC-TCXO) and Synchronous Ethernet (SyncE) Source

DC-TCXO is used now as the source of timing for SyncE, providing an accurate and stable clock for the synchronized operation of network devices that rely on Ethernet for timing.

DPA Application Signing

Allows DOCA applications signed with OEM/NVIDIA certificate private keys to be loaded onto the DPA engine, after the OEM/NVIDIA root certificates are installed on the NIC.

Data-Path Accelerator (DPA)

The DPA hardware version is now exposed as a new capability, labeled "dpa_platform_version."

Block SMP Traffic

Added a new NV config (SM_DISABLE, default 0) which, when enabled, blocks SMP traffic that does not originate from the SM.

Dynamic Long Cables

Added the ability to set cable length as a parameter in the PFCC access register. The cable length is used in the calculation of RX lossless buffer parameters, including size, Xoff, and Xon thresholds.

Bug Fixes

See Bug Fixes in this Firmware Version section.

Segment on PCIe Switch

Added support for Segment on PCIe switch.

AER on PCIe Switch Bridge

Added support for AER on PCIe switch bridge.

Bug Fixes

See Bug Fixes in this Firmware Version section.

General

This is the initial firmware release of NVIDIA® ConnectX®-8 SuperNIC.

ConnectX-8 has the same feature set as ConnectX-7 adapter card. For the list of the ConnectX-7 firmware features, please see ConnectX-7 Firmware Release Notes.

The features described here are new features in addition to the ConnectX-7 set.

Link Speed

NVIDIA® ConnectX®-8 SuperNIC supports 800Gb/s or XDR IB or 2 x 400GbE link speeds.

Note: 800GbE link speed is not supported on a single port.

Planarized Topology Network

ConnectX®-8 SuperNIC uses planarized topology network to reach Extended Data Rate (XDR) performance.

Direct NIC-GPU Datapath

To read/write data directly from the GPU and to overcome grace CPU PCIe bandwidth issue a direct NIC-GPU datapath is required.

To do so, the HCA exposes a side DMA engine as an additional PCIe function which is called “Data Direct”. This additional DMA engine allows vHCA access data buffers using MKEY through it, providing multiple PCIe data path interfaces. Such behavior is needed in a scenario where different memory region requires different PCIe data path, i.e NUMA (Non Uniform Memory Access) systems.

A vHCA is allowed to use a Data Direct function if HCA_CAP.data_direct is set. To use the Data Direct interface, the vHCA should create an MKEY with the data_direct bit set. The MKEY returned enables access through the side DMA engine. The MKEY access mode must be PA.

It supports only the following fields: a, rw, rr, lw, lr, relaxed_ordering_write, relaxed_ordering_read, mkey[7:0], length64, pd, start_addr, len. All other fields are reserved.

Congestion Control

Congestion Control provides performance isolation when multiple applications running on the same cluster. Additionally, it prevents congestion spreading when there is a slow receiver, reduce latency in the cluster, improves fairness, prevents parking-lot effects and packet's drop in lossy networks.

Multiple Encapsulation/Decapsulation Operation on a Packet

This capability enables the encapsulation table to be opened on both the FDB and the NIC tables together.

Crypto Algorithms

Extended the role-based authentication to cover all crypto algorithms. Now the TLS. IPsec. MACsec. GCM, mem2mem, and NISP work when nv_crypto_conf.crypto_policy = CRYPTO_POLICY_FIPS_LEVEL_2, meaning all cryptographic engines can also work in wrapped mode and not only in plaintext mode.

RoCE: Adaptive Timer

Enabled ADP timer to allow the user to configure RC or DC qp_timeout values lower than 16.

Multiple-Window in DPA Mode

Multi-window capability is now supported in DPA mode.

Doorbell Less QP

The new capability enables the user to send a queue without a doorbell record.

To create a doorbell less QP/SP, set send_dbr_mode = 1 in qp/sq ctx as defined in the PRM.

Packet's Flow Label Fields

The flow_label fields can be set, added or copied from the packet.

ODP Event

The following prefetch fields are available ODP event: pre_demand_fault_pages, post_demand_fault_pages

Jump from NIC_TX to FDB_TX

The user can jump from NIC_TX to FDB_TX table and bypass the ACL table using the 'table_type_valid' and 'table_type' fields available in the steering action (STC) "Jump To Flow" table.