Changes and New Feature History
This section includes history of changes and new feature of 3 major releases back. For older releases history, please refer to the relevant firmware versions.
OpenSNAPI Communication Channel
The communication channel is used to enable communication between processes on different vHCAs regardless of their network connectivity state.
Added a new NvConfig parameter LOG_MAX_QUEUE to set the maximum number of work queue resources (QP, RQ, SQ...) that can be created per function.
The default value is 2^17.
Congestion Control Key
Added a Congestion Control Key to all Congestion Control MADs to authenticate that they are originated from a trusted source.
Added an SMP firewall to block the option of sending SMPs (MADS sent on QP0 from the Subnet Manager) from unauthorized hosts to prevent fake SMPs from being recognized as the SM.
Vendor Specific MADs: Class 0x9
Vendor Specific MADs Class 0x9 is no longer supported by the firmware. If case the firmware detects such MAD, the firmware will return a "NOT SUPPORTED" error to the user.
Match Definer Object
Added support for a new steering match definer format (format 33).
Blocked the VF's ability to use both padding and signature in order to prevent the NIC from hanging.
Asserts' Severity Level
Added 3 new assert filters (Health buffer, NVlog, FW trace). The assert will be exposed now if its severity level is equal to or above the new filter.
The filters are configurable by the ini file. The "Health buffer" filter is also configurable by new access register.
Steering LAG Mode (Hash LAG)
[Beta] The new LAG mode (PORT_SELECT_FT LAG (hash LAG)) distributes the packets to ports according to the hash on the packet headers, instead of distributing the packets according to the QP (queue affinity – legacy LAG) to avoid cases where the slow/fast path packets are transmitted from different ports.
Identifying the right port is done by using destination type UPLINK with destination_eswitch_owner_vhca_id_valid set and destination_eswitch_owner_vhca_id indicating the PF associated with the port.
The below are the Queue Affinity and Steering LAG (hash) limitations:
Note: Due to changes in this feature, transmission timestamp in CQE is temporarily unsupported with multi eSwitch.
QSHR Access Register
Added support for QSHR access register to enable Set and Query rate limit per-host per-port.
Asymmetrical VFs per PF
Added support for asymmetrical VFs per PF.
To enable it:
mlxlink Support to read/write Access Registers by LID
Added 2 new MAD access registers to enable mlxlink to read/write access registers by LID (to the whole subnet).
See Bug Fixes.
Using NC-SI Commands for Debugging PCI Link Failures
Implemented a new NC-SI command get_debug_info to get mstdump via the NC-SI protocol to debug a device if the PCI link fails for any given reason.
Enable/Disable RDMA via the UEFI HII System Settings
Added support for Enabling/Disabling NIC and RDMA (port/partition) via the UEFI HII system settings.
Note: Values set in this option only take effect when is Ethernet mode.
NC-SI Speed Reporting
Updated the NC-SI speed reporting output to support 200GbE speed. Now when running the NC-SI command, the output presents 200GbE speed as well.
Increased the Maximum Number of MSIX per VF
Increased the maximum number of MSIX per VF to 127.
Asymmetrical MSIX Configuration
This feature allows the device to be configured with a different number of MSIX vectors per physical PCI functions.
To use this feature, please follow these steps:
Adaptive Routing (AR): multi_path, data_in_order
Added a new bit ("data_in_order") to query the QP and allow a process/library to detect when the AR is enabled.
flex_parser for GENEVE Hardware Offloadand ICMP
Added a new flex parser to support GENEVE hardware offloadand ICMP.
When the non-page-supplier-FLR funcion is initiated, the firmware triggers a page event to the page supplier to indicate that all pages should be returned for the FLR function.
Pages are returned by the driver to the kernel without issuing the MANAGE_PAGES commands to the firmare.
PCIe Eye Opening
Enabled measuring PCIe eye dynamic grading over PCIe Gen3 speed.
See Bug Fixes in this Firmware Version section.
Precision Time Protocol (PTP)
Enabled Precision Time Protocol (PTP) timestamp (UTC mode) in Multi-Host devices.
RoCE, Lossy, slow_restart_idle
Removed triggering unexpected internal CNPs for RoCE Lossy slow_restart_idle feature.
Added support for a new NV config mode “icm_cache_mode_large_scale_steering” that enables less cache misses and improves performance for cases when working with many steering rules.
Origin bit in the ctrl Segment of the WQE
This new capability adds an origin bit in the ctrl segment of the WQE sent by the Target to the Initiator during frontend controller exceptions in the NVMeoF interface. The WQE sent with the origin bit "set" will have SWQE_OPCODE_SEND_IMM opcode.
Connection Tracking Window Validation
Enabled connection tracking window validation by supporting CONN_TRACK_OFFLOAD general object and WQE with opcode ASO and opmode CONN_TRACK.
This new capability enables VF/VF-group rate-limiting while per-host rate-limiter is also applied.
Expanded the Sub-Function capability to support the scale of up to 512 by supporting more MSIX, removing internal obstacles and reducing the ICM memory footprint.
This new capability enables asymmetric Sub-Function configuration. It expands the asymmetric Sub-Function configuration to support asymmetric configurations between all PFs, and provides Sub-Function scalability and asymmetric NV configurations.
Match Definer Object
Added support for new steering match definer formats.
Steering by Sanity Check Flags
Added support for matching the following flags: l3_ok, l4_ok, ipv4_checksum_ok, l4_checksum_ok bits in create_flow_group and set_flow_table_entry commands, as well as in match definer formats (formats 0,1,2 and 3) to be used in matching actions on the packet headers.
See Bug Fixes section.
[Beta] This capability allows the software to reserve a QPN that can be used to establish connection performed over RDMA_CM, and provide the software a unique QP number.
Since RDMA_CM does not support DC, by using CREATE_QPN_RESERVED_OBJECT the software can reserve a QPN value from the firmware's managed QP number namespace range. This allows multiple software processes to hold a unique QPN value instead of using UD-QPs.
See Bug Fixes section.
Multi-Application QoS per QP
Added the option to allow applications to build their own QoS tree over the NIC hierarchy by connecting QPs to responder/requestor Queue Groups.
NRZ Link Performance
Improved NRZ link performance (RX algorithm).
NRZ Link-Up Time
Improved NRZ link-up time (25G\50G\100G speeds).
Direct Packet Placement (DPP)
Added support for Direct Packet Placement (DPP). DPP is a receive side transport service in which the Ethernet packets are scattered to the memory according to a packet sequence number (PSN) carried by the packet, and not by their arrival order. To enable DPP offload, the software should create a special RQ by using the CREATE_RQ command, and set DPP relevant attributes.
Ethernet wqe_too_small Mode
Added a new counter per vPort that counts the number of packets that reached the Ethernet RQ but cannot fit into the WQE due to their large size. Additionally, we added the option to control if such packet will cause “CQE with Error” or “CQE_MOCK”.
ignore_flow_level is now enabled by the TRUST LEVEL access registry.
Pause Frames from VFs
[Beta] Enabled the capability to allow Virtual Functions to send Pause Frames packets.
Added support for the cq_overrun counter. The counter represents the number of times CQs enter an error state due to overflow that occur when the device tries to post a CQE into a full CQ buffer.
Hardware steering dump output used for debugging and troubleshooting.
Please see Known Issue 2213356 for its limitations.
Sub Function (SF) BAR Size
Increased the minimum Sub Function (SF) BAR size from 128KB to 256KB. Due to the larger SF BAR size, for the same PF BAR2 size, which can be queried/modified by LOG_PF_BAR2_SIZE NV config, the firmware will support half of the SFs.
To maintain the same amount of supported SFs, software needs to increase the LOG_PF_BAR2_SIZE NV config value by 1.
GPUDirect in Virtualized Environment
Enabled a direct access to ATS from the NIC to GPU buffers using PCIe peer-to-peer transactions. To enable this capability, the “p2p_ordering_mode” parameter was added to the NV_PCI_CONF configuration.
Added a new Non-Volatile Configuration parameter to control VL15 buffer size (VL15_BUFFER_SIZE).
Note: VL15 buffer size enlargement will decrease all other VLs buffers size.
Added a new NC-SI command (get_device_id) to report a unique device identifier.
Added new NC-SI commands (get_lldp_nb, set_lldp_nb) to query the current status of LLDP and to enable/disable it.
Split the SlowRestart ROCE_ACCL into the following:
Enabled TX PSN window size configuration using LOG_TX_PSN_WINDOW NVconfig parameter.
Note: Due to hardware limitations, max log_tx_psn_win value can be set 9.
See Bug Fixes.
This is the initial firmware release of NVIDIA® Mellanox® ConnectX®-6 Lx adapter cards.
PHY-less Reset in PLU
[Beta] Enables the user to reset the firmware without resetting the PHY. The links that were up will stay up, all the other links will be disabled.
Note: Currently this capability does not support firmware upgrade and downgrade.
Hardware Tag Matching
Increased the maximum XRQ number to 512.
Updated the Packet Pacing clock to be in sync with the PTP clock.
Added support for hardware real time clock by UTC timestamp in PCIE BAR and CQE.
NC-SI 1.2 New Command
Implemented the following new command from NS-SI 1.2 specification:
Non-Volatile Configurations (NVCONFIG)
Added the following new mlxconfig parameters to the Non-Volatile Configurations section.