Changes and New Feature History

Added support for BMC with a single PF per NUMA in Socket-Direct adapter cards.

Note: SIngle PF per NUMA should not be enabled on Multi-host. 

Added a new NvConfig parameter LOG_MAX_QUEUE to set the maximum number of work queue resources (QP, RQ, SQ...) that can be created per function.

The default value is 2^17.

Added a Congestion Control Key to all Congestion Control MADs to authenticate that they are originated from a trusted source.

Added an SMP firewall to block the option of sending SMPs (MADS sent on QP0 from the Subnet Manager) from unauthorized hosts to prevent fake SMPs from being recognized as the SM.

The teardown of hotplugged emulated device (a.k.a unplug flow) is in the reverse order of the plug flow. However, certain legacy host software stack does not support surprise removal of the PCIe PF devices.

To support such host software stack, emulation manager software will perform a graceful teardown.

Added 3 new assert filters (Health buffer, NVlog, FW trace). The assert will be exposed now if its severity level is equal to or above the new filter.

The filters are configurable by the ini file. The "Health buffer" filter is also configurable by new access register.

Added support for clock frequency synchronization based on Synchronous Ethernet protocol.

Note: This capability is not supported with link speeds of 50G and higher, and cannot run in parallel with diagnostic counters.

Added support for: 

• Flow Direct
• LACP
• GRE Offload

[Beta] The new LAG mode (PORT_SELECT_FT LAG (hash LAG)) distributes the packets to ports according to the hash on the packet headers, instead of distributing the packets according to the QP (queue affinity – legacy LAG) to avoid cases where the slow/fast path packets are transmitted from different ports.

Identifying the right port is done by using destination type UPLINK with destination_eswitch_owner_vhca_id_valid set and destination_eswitch_owner_vhca_id indicating the PF associated with the port.

The below are the Queue Affinity and Steering LAG (hash) limitations: 

• Queue Affinity (legacy LAG) limitation:
  • LAG cannot be created when other functions on eSwitch are active (VFs, SFs, and x86 PF for SmartNIC).
    Note: This limitation does not exist in older firmware versions (xxx.31.1014 and below) when setting mlxconfig HIDE_PORT2_PF parameter for SmartNICs. This solution is no longer applicable when using firmware version xx.31.2xxx. 

• Steering LAG (hash) limitations:
  • LAG cannot be created when other functions on eSwitch are active (VFs, SFs, and x86 PF for smartnic) 
    Note: This limitation can be solved by setting LAG_RESOURCE_ALLOCATION=1
  • Cannot create PORT_SELECT_FT LAG when the SQs on PF (connected to strict TIS) are opened. 
    Note: This limitation can be solved by setting LAG_RESOURCE_ALLOCATION=1
    
    Note: It is recommended to set LAG_RESOURCE_ALLOCATION=1 before configuring the PORT_SELECT_FT lag. LAG_RESOURCE_ALLOCATION will pre-allocate needed firmware and hardware resources in order to CREATE_LAG with less limitations.
  • Can be used only with Software Steering due to a bug with strict SQs in Firmware Steering.

Note: Due to changes in this feature, transmission timestamp in CQE is temporarily unsupported with multi eSwitch.

Added support for asymmetrical VFs per PF. 

To enable it:
PF_NUM_OF_VF_VALID must be true, and PF_NUM_OF_VF to a none zero value.

Enabled the option to dynamically modify the MSIX and the number of virtio VF device queues.

Note: This modification must be done before loading the driver on the device. 

This new capability includes the following limitations:

• total queue/msix number can not exceed 2k
• queue/msix per virtio vf device cannot exceed 64
• the scale of virtio device is limit to below 127 from mlxconfig

Added support for Enabling/Disabling NIC and RDMA (port/partition) via the UEFI HII system settings.

Note: Values set in this option only take effect when is Ethernet mode.

Allows the device to be configured with more than 256 MSIX vectors per physical PCI function.

To use it, please follow the Asymmetrical MSIX configuration (PF_NUM_PF_MSIX_VALID) (see feature Asymmetrical MSIX Configuration below).

Note: When using this feature while Sub-Function configuration (PF_BAR2_ENABLE / PER_PF_NUM_SF) is enabled, it is recommended to configure “above 4G-decoding” in the BIOS due to device BAR space requirements.

Allows the device to be configured with a different number of MSIX vectors per physical PCI functions.

To use this feature, please follow these steps:

1. Clear NUM_PF_MSIX_VALID to disable global symmetrical MSIX configuration.
2. Set PF_NUM_PF_MSIX_VALID to enable asymmetrical per Physical Function MSIX configuration.
3. Configure PF_NUM_PF_MSIX per physical PCI function.

Notes:

• When using this configuration, NUM_PF_MSIX (the symmetrical MSIX configuration field) will be ignored by the firmware although it will be present in the mlxconfig query.
• The asymmetrical MSIX configuration and the legacy symmetrical MSIX configuration are mutually exclusive. When both are enabled, the firmware will work according to the symmetrical MSIX configuration.
• Step #3 should be done on each of the physical PCI functions, otherwise, the device will use default configurations.

When the non-page-supplier-FLR funcion is initiated, the firmware triggers a page event to the page supplier to indicate that all pages should be returned for the FLR function.

Pages are returned by the driver to the kernel without issuing the MANAGE_PAGES commands to the firmare.

Added support for new steering match definer formats.

• Match definer format 26
• Match definer format 27

[Beta] Enabled remote (PMA) loopback in NRZ, Rx-to-Tx.

Note: To use the PMA loopback, both sides should be in Force Mode (AN Disabled).

[Beta] This capability allows the software to reserve a QPN that can be used to establish connection performed over RDMA_CM, and provide the software a unique QP number.

Since RDMA_CM does not support DC, by using CREATE_QPN_RESERVED_OBJECT the software can reserve a QPN value from the firmware's managed QP number namespace range. This allows multiple software processes to hold a unique QPN value instead of using UD-QPs.

Hardware steering dump output used for debugging and troubleshooting.

Please see Known Issue 2213356 for its limitations.

Increased the minimum Sub Function (SF) BAR size from 128KB to 256KB. Due to the larger SF BAR size, for the same PF BAR2 size, which can be queried/modified by LOG_PF_BAR2_SIZE NV config, the firmware will support half of the SFs.

To maintain the same amount of supported SFs, software needs to increase the LOG_PF_BAR2_SIZE NV config value by 1.

Added a new Non-Volatile Configuration parameter to control VL15 buffer size (VL15_BUFFER_SIZE).

Note: VL15 buffer size enlargement will decrease all other VLs buffers size.

Split the SlowRestart ROCE_ACCL into the following:

• slow-restart – used to reduce rate on retransmission events
• slow-restart-after-idle – used to reduce rate before first transmission after >1s without transmitting

Enabled TX PSN window size configuration using LOG_TX_PSN_WINDOW NVconfig parameter.

Note: Due to hardware limitations, max log_tx_psn_win value can be set 9.

Enabled with IPsec transport mode (ESP header over IP header). All the bellow settings are done by the hardware (full offload):

• packet re-formatting, adding/deleting ESP header/trailer, and SN configuration
• encryption and decryption

Implemented the following new command from NS-SI 1.2 specification:

• Get PF Assignment

ECE is new negotiation scheme used to exchange extra information about QPs capabilities and later negotiate them at the connection establishment phase. It is used to support various features such as RoCE Selective Repeat and PCC.

RoCE Selective Repeat activation now is available using ECE. This feature is enabled automatically using ECE if NV_config.sr_enable parameter is set 1 otherwise using go back N retransmission.

This new capability comes with the following limitations:

• Selective Repeat results in packets arriving out of order
• Signature is not supported with Selective Repeat.
• Need to create mkey with signature before QPs are opened in gvmi

Added the following new mlxconfig parameters to the Non-Volatile Configurations section.

• log_max_outstandng_wqe
• ece_disable_mask
• NV_config.sr_enable

The following are the minimal software/firmware versions that support PAM4 link speeds when connected using Mellanox NIC to Mellanox Switch and Mellanox NIC to 3rd Party Switches:

• Mellanox Spectrum-3: 30.2007.1142
• Mellanox Spectrum-2: 29.2007.1142
• Switch SDK: 4.4.0920
• Mellanox Onyx: 3.9.0830-038
• SONiC/SAI: 201911
• ConnectX-6 Dx: 22.27.2008*

*Note: NICs with this firmware version support Mellanox-to-Mellanox connectivity with PAM4 link speeds

Added the following segments, as appeared in the PRM, to the Resource Dump:

• PRM_QUERY_QP
• PRM_QUERY_CQ
• PRM_QUERY_MKEY
• QUERY_VNIC_ENV