Configuring the Driver Registry Keys

Linux Kernel Upstream Release Notes v6.5

NVIDIA® IPoIB and Ethernet drivers use registry keys to control the NIC operations. The registry keys receive default values during the installation of the NVIDIA® adapters. Most of the parameters are visible in the registry by default, however, certain parameters must be created in order to modify the default behavior of the NVIDIA® driver.
The adapter can be configured either from the User Interface (Device Manager -> Mellanox Adapter -> Right click -> Properties) or by setting the registry directly.
All NVIDIA® adapter parameters are located in the registry under the following registry key:

Copy
Copied!
            

HKEY_LOCAL_MACHINE \SYSTEM \CurrentControlSet \ Control \ Class \{4D36E972-E325-11CE-BFC1-08002bE10318} \<Index>

The registry key can be divided into 4 different groups:

Group

Description

Basic

Contains the basic configuration.

Offload Options

Controls the offloading operation that the NIC supports.

Performance Options

Controls the NIC operation in different environments and scenarios.

Flow Control Options

Controls the TCP/IP traffic.

Any registry key that starts with an asterisk ("*") is a well-known registry key. For more details regarding the registries, please refer to:
http://msdn.microsoft.com/en-us/library/ff570865(v=VS.85).aspx

Procedure_Heading_Icon.PNG

To find the index value of your Network Interface from the Device Manager please perform the following steps:

  1. Open Device Manager, and go to Network Adapters.

  2. Right click ->Properties on Mellanox Connect-X® Ethernet Adapter.

  3. Go to Details tab.

  4. Select the Driver key, and obtain the nn number.

In the below example, the index equals 0010.

Network_Interface_Index_Value.png

Warning

All registry keys added for driver configuration should be of string type (REG_SZ).

Warning

After setting a registry key and re-loading the driver, you may use the mlx5cmd -regkeys command to assure that the value was read by the driver.

This group contains the registry keys that control the basic operations of the NIC

Value Name

Default Value

Description

*JumboPacket

ETH: 1514

IPoIB: 4092

The maximum size of a frame (or a packet) that can be sent over the wire. This is also known as the maximum transmission unit (MTU). The MTU may have a significant impact on the network's performance as a large packet can cause high latency. However, it can also reduce the CPU utilization and improve the wire efficiency. The standard Ethernet frame size is 1514 bytes, but NVIDIA® drivers support wide range of packet sizes.

The valid values are:

  • Ethernet: 614 up to 9614

  • IPoIB: 600 up to 4092

Note: All the devices across the network (switches and routers) should support the same frame size. Be aware that different network devices calculate the frame size differently. Some devices include the header, i.e. information in the frame size, while others do not.

NVIDIA® adapters do include Ethernet header information in the frame size. (i.e when setting *JumboPacket to 1514, the actual payload size is 1500 bytes).

*ReceiveBuffers

512

The number of packets each ring receives. This parameter affects the memory consumption and the performance. Increasing this value can enhance receive performance, but also consumes more system memory.

In case of lack of received buffers (dropped packets or out of order received packets), you can increase the number of received buffers.

The valid values are 256 up to 4096.

*TransmitBuffers

2048

The number of packets each ring sends. Increasing this value can enhance transmission performance, but also consumes system memory.

The valid values are 256 up to 4096.

*NetworkDirect

1

The *NetworkDirect keyword determines whether the miniport driver's NDK functionality can be enabled. If this keyword value is set to 1 ("Enabled"), NDK functionality can be enabled. If it is set to 0 ("Disabled"), NDK functionality cannot be enabled.

Note: This key is enabled by default, thus NDK will be used. It is important to set the switch to enable ECN and/or PFC otherwise the system will experience performance degradation.

Note: This key affects NDK functionality and not Userspace ND (Network Direct).

For further details, see: https://msdn.microsoft.com/en-us/windows/hardware/drivers/ network/enabling-and-disabling-ndk-functionality

*NetworkDirectTechnology

0

The *NetworkDirectTechnology keyword determines the technology used for the device.

The valid values are:

  • 0 - Device Default

  • 3 - RoCE

  • 4 - RoCE v2

For further details, see: https://docs.microsoft.com/en-us/windows-hardware/drivers/network/inf-requirements-for-ndkpi

Key Name

Key Type

Values

Description

ThreadedDpcEnable

DWORD

  • 0 - Disabled

  • 1 - Enabled

Controls the threaded DPC mode enablement for Rx traffic completion processing.

TxThreadedDpcEnable

DWORD

  • 0 - Disabled

  • 1 - Enabled

Controls the threaded DPC mode enablement for Tx traffic completion processing.

CheckForHangTOInSeconds

REG_DWORD

[0 – MAX_ULONG]

Default: 4

The interval in seconds for the Check- for-Hang mechanism

Note: This registry key is available only when using WinOF-2 v2.0 and later.

Note: As of WinOF-2 v2.10, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

This group of registry keys allows the administrator to specify which TCP/IP offload settings are handled by the adapter rather than by the operating system.

Enabling offloading services increases transmission performance. Due to offload tasks (such as checksum calculations) performed by adapter hardware rather than by the operating system (and, therefore, with lower latency). In addition, CPU resources become more available for other tasks.

Value Name

Default Value

Description

*LsoV2IPv4

1

Large Send Offload Version 2 (IPv4). The valid values are:

  • 0: disable

  • 1: enable

*LsoV2IPv6

1

Large Send Offload Version 2 (IPv6). The valid values are:

  • 0: disable

  • 1: enable

LSOSize

64000

The maximum number of bytes that the TCP/IP stack can pass to an adapter in a single packet.

This value affects the memory consumption and the NIC performance. The valid values are MTU+1024 up to 64000.

Note: This registry key is not exposed to the user via the UI. If LSOSize is smaller than MTU+1024, LSO will be disabled.

LSOMinSegment

2

The minimum number of segments that a large TCP packet must be divisible by, before the transport can offload it to a NIC for segmentation. The valid values are 2 up to 32.

Note: This registry key is not exposed to the user via the UI.

LSOTcpOptions

1

Enables that the miniport driver to segment a large TCP packet whose TCP header contains TCP options.

The valid values are:

  • 0: disable

  • 1: enable

Note: This registry key is not exposed to the user via the UI.

LSOIpOptions

1

Enables its NIC to segment a large TCP packet whose IP header contains IP options.

The valid values are:

  • 0: disable

  • 1: enable

Note: This registry key is not exposed to the user via the UI.

*IPChecksumOffloadIPv4

3

Specifies whether the device performs the calculation of IPv4 checksums. The valid values are:

  • 0: (disable)

  • 1: (Tx Enable)

  • 2: (Rx Enable)

  • 3: (Tx and Rx enable)

*TCPUDPChecksumOffl oadIPv4

3

Specifies whether the device performs the calculation of TCP or UDP checksum over IPv4.

The valid values are:

  • 0: (disable)

  • 1: (Tx Enable)

  • 2: (Rx Enable)

  • 3: (Tx and Rx enable)

*TCPUDPChecksumOffl oadIPv6

3

Specifies whether the device performs the calculation of TCP or UDP checksum over IPv6.

The valid values are:

  • 0: (disable)

  • 1: (Tx Enable)

  • 2: (Rx Enable)

  • 3: (Tx and Rx enable)

*RssOnHostVPorts

1

Virtual Machine Multiple Queue (VMMQ) HW Offload The valid values are:

  • 0: disable

  • 1: enable

SwParsing

N/A

Specifies whether the device performs the calculation of TCP checksum over IP-in-IP encapsulated IPv4/6 sent packets.

The valid values are:

  • 0: (disable)

  • 1: (Tx Enable)

UsoIPv4

1

UDP Segmentation Offload (IPv4).

The valid values are:

  • 0: (Disable)

  • 1: (Enable)

UsoIPv6

1

UDP Segmentation Offload (IPv6).

The valid values are:

  • 0: (Disable)

  • 1: (Enable)

This group of registry keys configures parameters that can improve adapter performance.

Value Name

Default Value

Description

TxIntModerationProfile

1

Enables the assignment of different interrupt moderation profiles for send completions. Interrupt moderation can have great effect on optimizing network throughput and CPU utilization.

The valid values are:

  • 0: Low Latency

    Implies higher rate of interrupts to achieve better latency, or to handle scenarios where only a small number of streams are used.

  • 1: Moderate

    Interrupt moderation is set to midrange defaults to allow maximum throughput at minimum CPU utilization for common scenarios.

  • 2: Aggressive

    Interrupt moderation is set to maximal values to allow maximum throughput at minimum CPU utilization for more intensive, multi-stream scenarios.

  • 3: Dynamic

    Improve existing system performance by changing interrupt moderation dynamically while also decreasing latency and CPU usage

Note: As of WinOF-2 v2.10, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

RxIntModerationProfile

1

Enables the assignment of different interrupt moderation profiles for receive completions. Interrupt moderation can have a great effect on optimizing network throughput and CPU utilization.

The valid values are:

  • 0: Low Latency

    Implies higher rate of interrupts to achieve better latency, or to handle scenarios where only a small number of streams are used.

  • 1: Moderate

    Interrupt moderation is set to midrange defaults to allow maximum throughput at minimum CPU utilization for common scenarios.

  • 2: Aggressive

    Interrupt moderation is set to maximal values to allow maximum throughput at minimum CPU utilization, for more intensive, multi-stream scenarios.

  • 3: Dynamic

    Improve existing system performance by changing interrupt moderation dynamically while also decreasing latency and CPU usage

Note: As of WinOF-2 v2.10, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

RecvCompletionMethod

1

Sets the completion methods of the receive packets, and it affects network throughput and CPU utilization.

The supported methods are:

  • Polling - increases the CPU utilization, because the system polls the received rings for incoming packets; however, it may increase the network bandwidth since the incoming packet is handled faster.

  • Adaptive - combines the interrupt and polling methods dynamically, depending on traffic type and network usage.

The valid values are:

  • 0: polling

  • 1: adaptive

*InterruptModeration

1

Sets the rate at which the controller moderates or delays the generation of interrupts, making it possible to optimize network throughput and CPU utilization. When disabled, the interrupt moderation of the system generates an interrupt when the packet is received. In this mode, the CPU utilization is increased at higher data rates, because the system must handle a larger number of interrupts. However, the latency is decreased, since that packet is processed more quickly.

When interrupt moderation is enabled, the system accumulates interrupts and sends a single interrupt rather than a series of interrupts.

The valid values are:

  • 0: disable

  • 1: enable

Note: As of WinOF-2 v2.10, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

RxIntModeration

2

Sets the rate at which the controller moderates or delays the generation of interrupts, making it possible to optimize network throughput and CPU utilization. The default setting (Adaptive) adjusts the interrupt rates dynamically, depending on traffic type and network usage. Choosing a different setting may improve network and system performance in certain configurations.

The valid values are:

  • 0: off

  • 1: static

  • 2: adaptive

  • 3: dynamic

The interrupt moderation count and time are configured dynamically, based on traffic types and rate.

TxIntModeration

4 - Default

Sets the rate at which the controller moderates or delays the generation of interrupts, making it possible to optimize network throughput and CPU utilization. The default setting (Adaptive) adjusts the interrupt rates dynamically, depending on traffic type and network usage. Choosing a different setting may improve network and system performance in certain configurations.

The valid values are:

  • 0: off

  • 1: static

  • 2: adaptive

  • 3: dynamic

  • 4: default

The interrupt moderation count and time are configured dynamically, based on traffic types and rate.

Default mode (4) will set Adaptive (2) for RSS mode setup configuration, and OFF (0) in all other cases.

*RSS

1

Sets the driver to use Receive Side Scaling (RSS) mode to improve the performance of handling incoming packets. This mode allows the adapter port to utilize the multiple CPUs in a multi-core system for receiving incoming packets and steering them to their destination. RSS can significantly improve the number of transactions per second, the number of connections per second, and the network throughput.

This parameter can be set to one of two values:

  • 1: enable (default)

    Sets RSS Mode.

  • 0: disable

    The hardware is configured once to use the Toeplitz hash function and the indirection table is never changed.

ThreadPoll

3000

The number of cycles that should be passed without receiving any packet before the polling mechanism stops when using polling completion method for receiving. Afterwards, receiving new packets will generate an interrupt that reschedules the polling mechanism.

The valid values are 0 up to 200000.

Note: This registry value is not exposed via the UI.

Note: As of WinOF-2 v2.10, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

VlanId

ETH: 0

Enables packets with VlanId. It is used when no team intermediate driver is used.

The valid values are:

  • 0: disable

    No VLAN Id is passed.

  • 1-4095

    Valid VLAN ID that will be passed.

Note: This registry value is only valid for Ethernet.

*NumRSSQueues

8

The maximum number of the RSS queues that the device should use.

Note: This registry key is only in Windows Server 2012 and above.

BlueFlame

1

The latency-critical Send WQEs to the device. When a BlueFlame is used, the WQEs are written directly to the PCI BAR of the device (in addition to memory), so that the device may handle them without having to access memory, thus shortening the execution latency. For best performance, it is recommended to use the BlueFlame when the HCA is lightly loaded. For high- bandwidth scenarios, it is recommended to use regular posting (without BlueFlame).

The valid values are:

  • 0: disable

  • 1: enable

Note: This registry value is not exposed via the UI.

*MaxRSSProcessors

8

The maximum number of RSS processors.

Note: This registry key is only in Windows Server 2012 and above.

AsyncReceiveIndicate

0

Disabled default

1

Enables packet burst buffering using threaded DPC

2

Enables packet burst buffering using polling

RfdReservationFactor

150

Controls the number of reserved receive packets,

*RscIPv4

1

Enable or disable support for RSC for the IPv4 datagram version.

*RscIPv6

1

Enable or disable support for RSC for the IPv6 datagram version.

MaxCallsToNdisIndicate

5

Maximum number of times chained packets can be indicated before packets processing is stop processing is stopped.

RssV2

0

Enables the RSS v2 feature which improves the Receive Side Scaling by offering dynamic, per-VPort spreading of queues. It reduces the time to update the indirection table.

Note: RSSv2 is only supported by NDIS 6.80 and later versions.

ValidateRssV2

0

Enables strict argument validation for upper layer testing. Set along with the RssV2 key to enable the RSSv2 feature.

StridingRqEnabled

0

When set, enables the Striding RQ feature. The receive buffers are segmented into fixed size strides and each incoming packet (or an LRO aggregate) consumes a buffer of its size.

NumberOfStrides

16

Relevant when Striding RQ feature is enabled. The value can be power of two in the range 8-256,. This value will determine the number of segments of receive buffer.

In General, Receive buffer size is determined by the maximum between RscMaxPacketSize and *JumboPacket. (for this value we might add headers or additional alignments required by HW).

The buffer size is divided into NumberOfStrides segments. Each segment size can be of range 64-8192.

In case of inconsistency with those values, the following Event log message will be displayed:

MLX_EVENT_LOG_ILLEGAL_STRIDE_RQ_PARAM will appear and Receive buffers will not be segmented.

All values can be seen via tool using command: mlx5Cmd -Stat -Verbose

*RscIPv4

unset

When set to ‘1’ LRO is enabled.

*RscIPv6

unset

When set to ‘1’ LRO is enabled.

RscMaxPacketSize

unset

In this configuration, this value should be from 16KB up

to 64KB (64*1024).

EnableZtt

0x0 (Disable)

ZTT register enables users to configure the device zero touch tuning algorithm.

0x0: Disable (Default)

0x1: Enable

The following section describes the registry keys that are only relevant to Ethernet driver.

Value Name

Default Value

Description

RoceFrameSize

Unset

(Will be derived from JumboPacket)

The maximum size of a frame (or a packet) that can be sent by the RoCE protocol (a.k.a Maximum Transmission Unit (MTU).

Using larger RoCE MTU will improve the performance; however, one must ensure that the entire system, including switches, supports the defined MTU.

Ethernet packet uses the general MTU value, whereas the RoCE packet uses the RoCE MTU.

When defining the RoCE MTU, the size of the JumboPacket should be taken into consideration. The value must be set according to the following formula:

JumboPacket >= RoCE_MTU + Header

The valid values are:

  • 256

  • 512

  • 1024

  • 2048

  • 4096

Note: This registry key is supported only in Ethernet drivers.

*PriorityVLANTag

3: Packet Priority & VLAN Enabled

Enables sending and receiving IEEE 802.3ac tagged frames, which include:

  • 802.1p QoS (Quality of Service) tags for priority-tagged packets.

  • 802.1Q tags for VLANs.

When this feature is enabled, the NVIDIA® driver supports sending and receiving a packet with VLAN and QoS tag.

DeviceRxStallTimeout

1000

The maximum period for a single received packet processing. If the packet was not processed during this time, the device will be declared as stalled and will increase the "Critical Stall Watermark Reached" counter. The value is given in mSec. The maximum period is 8000 mSec. The special value of 0, indicates that the DeviceRxStallTimeout is active.

Range: 0x0050 (80)- 0x1F40 (8000)

Note: As of WinOF-2 v2.20, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

DeviceRxStallWatermark

0

The maximum period for a single received packet processing. If the packet was not processed during this time, the device will increase a diagnostic counter called "Minor Stall Watermark Reached”. The value is given in mSec. The maximum period is 8000 mSec. The special value of 0 indicates that the DeviceRxStallWatermark is active

Range: 0x0050 (80)- 0x1F40 (8000)

Note: As of WinOF-2 v2.20, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

TCHeadOfQueueLifeTimeLimit

0-20

Default: 19

The time a packet can live at the head of a TC queue before it is discarded. The timeout value is defined by 4,096us multiplied by 2^TCHeadOfQueueLifetimeLimit.

Note: As of WinOF-2 v2.20, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

TCHeadOfQueueLifeTimeLimitEnable

0-255

Default: 255

Enables the TCHeadOfQueueLifeTimeLimit.

Note: As of WinOF-2 v2.20, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

TCStallCount

1-7

Default: 1

The number of sequential packets dropped due to Head Of Queue Lifetime Limit, in order for the port to enter the TCStalled state. All packets for the TC are discarded in this state for a period of 8 times the timeout defined by TCHeadOfQueueLifetimeLimit.

Note: As of WinOF-2 v2.20, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

TCStallEnable

0 - Disabled

1 - Enabled (Default)

Enables/Disables the TCStalled state.

Note: As of WinOF-2 v2.20, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

TCHeadOfQueueLifeTim eLimitEnable

0

The TCs for which Head Of Queue Lifetime Limit is enabled. Bit 0 represents TC0, bit 1 represents TC1 and so on.

The valid values are:

  • 0-255

  • 0: disabled

Note: As of WinOF-2 v2.20, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

RelaxedOrderingWrite

0 - Disabled

1 - Enabled

Default: Auto Detect

When this register is set, a PCIe cycle is issued with "relaxed ordering" attribute (allows write after write bypassing) for writes.

Note: This register is supported only in Ethernet flows and not RDMA.

For additional information on the PCIe relaxed ordering feature please refer to the PCI Express® Base Specification section on Transaction Ordering Rules.

Default value is Auto Detect, meaning the feature is enabled always unless the CPU family is Haswell or Broadwell where the feature will be disabled as a performance degradation is expected.

VFAllowedRelaxedOrdering

0 - No Relaxed Ordering will be supported for new VFs

1 - Only Relaxed Ordering Write will be supported for new VFs

2 - Only Relaxed Ordering Read will be supported for new VFs

3 - Both Relaxed Ordering types will be supported for new VFs (Default)

Limits the PCIe relaxed ordering feature for VFs.

Note: When set to 0, limitation is disabled. Although the key is dynamic, changes will take effect after VFs are created. For additional information on the PCIe relaxed ordering feature please refer to the PCI Express® Base Specification section on Transaction Ordering Rules.

Note: This registry key cannot be changed in Bluefield 2 SmartNIC mode, the value in this setup will be 3.

DisableLocalLoopbackFlags

0 - Do not disable any local loopback (Default)

1 - Disable Multicast

2 - Disable Unicast

3 - Disable Unicast and Multicast

This key controls whether or not to disable any local Loopback.

Flow Control Options

This group of registry keys allows the administrator to control the TCP/IP traffic by pausing frame transmitting and/or receiving operations. By enabling the Flow Control mechanism, the adapters can overcome any TCP/IP issues and eliminate the risk of data loss.

Value Name

Default Value

Description

*FlowControl

3

When Rx Pause is enabled, the receiving adapter generates a flow control frame when its received queue reaches a pre-defined limit. The flow control frame is sent to the sending adapter.

When TX Pause is enabled, the sending adapter pauses the transmission if it receives a flow control frame from a link partner.

The valid values are:

  • 0: Flow control is disabled

  • 1: Tx Flow control is Enabled

  • 2: Rx Flow control is enabled

  • 3: Rx & Tx Flow control is enabled

DeviceRxStallTimeout

1000 mSec

When the device is in stall state (congestion mode), after the configured period of having the device in such state expires (the maximum period is 8 sec), the device will disable the Flow Control mechanism.

The valid values are:

  • Minimum: 0

  • Maximum: 8000

DeviceRxStallWatermark

0 mSec

When the device is in "stall state" (congestion mode), after the configured period of having the device in such state expires (the maximum period is 8 sec), the device will declare the driver as stalled.

The valid values are:

  • Minimum: 0

  • Maximum: 8000


VMQ Options

This section describes the registry keys that are used to control the NDIS Virtual Machine Queue (VMQ). VMQ is supported by WinOF-2 and allows a performance boost for Hyper-V VMs.

For more details about VMQ please refer to Microsoft web site, http://msdn.microsoft.com/en-us/library/windows/hardware/ff571034(v=vs.85).aspx

Value Name

Default Value

Description

*VMQ

1

The support for the virtual machine queue (VMQ) features of the network adapter.

The valid values are:

  • 1: enable

  • 0: disable

*RssOrVmqPreference

0

Specifies whether VMQ capabilities should be enabled instead of receive- side scaling (RSS) capabilities.

The valid values are:

  • 0: Report RSS capabilities

  • 1: Report VMQ capabilities

Note: This registry value is not exposed via the UI.

*VMQVlanFiltering

1

Specifies whether the device enables or disables the ability to filter network packets by using the VLAN identifier in the media access control (MAC) header.

The valid values are:

  • 0: disable

  • 1: enable


RoCE Options

This section describes the registry keys that are used to control RoCE mode.

Value Name

Default Value

Description

roce_mode

0 - RoCE

The RoCE mode. The valid values are:

  • 0 - RoCE

  • 4 - No RoCE

Note: The default value depends on the WinOF package used.


SR-IOV Options

This section describes the registry keys that are used to control the NDIS Single Root I/O Virtualization (SR-IOV). The SR-IOV is supported by WinOF-2 and allows a performance boost for Hyper-V VMs.

For more details about the VMQ, please see Single Root I/O Virtualization (SR-IOV) on Microsoft website.

Value Name

Default Value

Description

*SRIOV

1

The support for the SR-IOV features of the network adapter. The valid values are:

  • 1: enable

  • 0: disable

*SriovPreferred

N/A (hidden)

A value that defines whether SR-IOV capabilities should be enabled instead of the virtual machine queue (VMQ), or receive side scaling (RSS) capabilities.

MaxFWPagesUsagePerVF

250000

This key sets the limitation for the maximum number of 4KB pages that the host could allocate for VFs resources. When set to 0, limitation is disabled.

The minimum valid value (when it is not 0) is 17000. When a smaller value (and larger than 0) is configured, the driver will use 17000 instead of the configured value.

Note: This key can be changed dynamically.


The following section describes the registry keys that are only relevant to RDMA.

Value Name

Default Value

Description

EnableGuestRdma

1: Enabled

Able to prevent RDMA in the VF from the host. This feature is enabled by default in IPoIB.

Note: This registry key cannot be changed in Bluefield 2 SmartNIC mode, the selected mode in this setup will be enabled.

MaxCMRetries

15

Maximum number of times that either party can re-send a REQ, REP, or

DREQ message. After re-sending for the maximum number of times without a response, the sending party should then terminate the protocol

by sending a REJ message indicating that it timed out.

RemoteCMResponseTimeout

16

Expressed as 4.096 microSec * 2 ^ cm_response_timewait, within which the CM message recipient shall transmit a response to the sender.

Valid values are: 3-25

NetworkDirectAdminOnly

0

In case this key is set 1, only an Admin user can use the ND - NetworkDirect application.

Max value: 1

This registry key can be found at: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\ mlx5\Parameters

NdkFmrDedicatedQp

0: Disabled

Controls whether or not a separated QP is used for NDK fast-register operations.

EnableCmAntiSpoofing

0

In case this key is set 1, the CM will not accept connection requests which the source IP-in-IP header is different from the source IP-in-CM private date (if there is any difference, the connection will be refused).

Max value: 1

Dump Me Now (DMN) Registry Keys

The registry keys for the DMN feature are located at: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\{4d36e972-e325-11ce- bfc1-08002be10318}\nnnn\

For further information on how to find the registry keys, refer to section Finding the Index Value of the Network Interface.

The following section describes the registry keys that configure the Dump Me Now feature (see section Dump Me Now (DMN)).

Value Name

Key Type

Description

DumpMeNowDirectory

REG_SZ

Path to the root directory in which the DMN places its dumps. The path should be provided in a kernel path style, which means prefixing the drive name with "\??\" (e.g. \??\C:\DMN_DIR). BDF will be added to specified name. (e.g. if specified directory name is \??\C:\DMN_DIR, then directory \??\C:\DMN_DIR-<b>-<d>-<f> will be created for Host and \??\C:\DMN_DIR-<b>-<d> for VF)

Default Value:

  • Host: \Systemroot\temp\Mlx5_Dump_Me_Now-<b>-<d>-<f>

  • VF: \Systemroot\temp\Mlx5_Dump_Me_Now-<b>-<d>-0

DumpMeNowTotalCount

REG_DWORD

The maximum number of allowed DMN dumps. Newer dumps beyond this number will override old ones.

Values: [0,512]

Default Value: 128

Note: As of WinOF-2 v2.10, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

DumpMeNowPreservedCount

REG_DWORD

Specifies the number of DMN dumps that will be reserved, and will never be overridden by newer DMN dump.

Values: [0,512]

Default Value: 8

Note: As of WinOF-2 v2.10, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

DumpMeNowDumpMask

0xFFFF

Mask that controls the allowed dumps by DumpMeNow (If applicable).

  • MST_DUMP = 1 << 0

  • CORE_DUMP = 1 << 1

  • ADAPTER_DUMP = 1 << 2

  • PDDR_DUMP = 1 << 3

  • MP_STATS_DUMP = 1 << 4

  • MP_RESOURCE_DUMP = 1 << 5

  • REGISTRY_DUMP = 1 << 6

  • QoS_DUMP = 1 << 7

  • IPoIB_DUMP = 1 << 8

  • VMQoS_DUMP = 1 << 9

  • FULL_DUMP = 0xFFFF

Values: [0,0xFFFF]

Note: This key can be changed dynamically.

Warning

Setting DumpMeNowTotalCount and DumpMeNowPreservedCount to "0" will disable the DMN feature.


ResourceDump Registry Keys

The following section describes the registry keys that configure the ResourceDump feature (see section Resource Dump).

Value Name

Key Type

Description

ResourceDumpEnable

REG_DWORD

  • 0 - ResourceDump notifications are disabled

  • 1 - ResourceDump notifications are enabled

Values: [0,1]

Default Value: 0

Note: This key can be changed dynamically.

ResourceDumpQuotaTimeLimit

REG_DWORD

This key is used to manage the quota time in seconds, when the time passes this value, the quota count will be reset. This mechanism is to control how many events per the “Key Value” in seconds are allowed.

Values: [1, ‭1048575‬]

Default value: 3600 (1 hour)

Note: This key can be changed dynamically.

ResourceDumpQuotaCount

REG_DWORD

Quota Count in the period of QuotaTimeLimit are allowed.

Values: [1, 100]

Default Value: 5

Note: This key can be changed dynamically


FwTrace Registry Keys

The following section describes the registry keys that configure the FwTrace feature (see section FwTrace).

Value Name

Key Type

Description

FwTracerEnabled

REG_DWORD

  • 0 - FwTrace is disabled

  • 1 - FwTrace is enabled

Values: [0,1]

Default Value: 1

Note: As of WinOF-2 v2.10, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

FwTracerBufferSize

REG_DWORD

FwTracer Buffer Size in Bytes. This value is rounded up to be equal to 2^N * 4096 bytes.

Values: [0x2000, 0x200000]

Default Value: 0x10000

Note: As of WinOF-2 v2.10, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.


DevX Registry Keys

The following section describes the registry keys that configure the DevX feature (see section DevX Interface).

Value Name

Key Type

Description

DevxEnabled

REG_DWORD

Enables Devx support.

  • 0 - disabled

  • 1 - enabled

Default Value: 0

DevxFsRules

REG_DWORD

Devx steering rules support (mask value).

  • 0 - Only the default IPV4/UDP DevX steering rule is supported

  • 8 - Add support for Unicast MAC DevX steering rule

  • 16 - Add support for IPV4/UDP with CVLAN DevX steering rule

  • 32 - Add support for promiscuous mode

  • 64 - Add support for IPv6 multicast

  • 128 - Add support for IPv6 and L4 protocol

  • 256 - Add support for Ethernet MAC and L4 protocol

  • 512 - Add support for IPv6 multicast and any IP version

  • 1024 - Add support for Ethernet MAC and any IP version

  • 2048 - Add support for L4 protocol

  • 4096 - Add support for any IP protocol

  • 8192 - Add support for IPv4 and TCP port with cvlan

  • 16384 - Add support for Ethernet MAC and VLAN

  • 32768 - Add support for IPv6 multicast and VLAN

  • 65536 - Add support for Ethernet MAC, VLAN and IP

  • 131072 - Add support for IPv6 multicast, VLAN and IP

  • 262144 - Add support for all multicast

AllowPromiscVport

REG_DWORD

Allows promiscuous mode enablement for vPorts.

  • 0 - Not allowed

  • 1 - Allowed

Default Value: 0

Note: This capability is not supported in BlueField DPU mode, VPORTs are controlled by the DPU side.


VF Monitoring Registry Keys

The following section describes the registry keys that configure the VF Monitoring feature (see section “VF Monitoring”).

The keys are located in the driver key of the adapter, and they are all dynamic.

Value Name

Key Type

Description

VfCpuMonEnable

REG_DWORD

The wanted state of the feature.

Values: [0,1]

Default Value: 0

Note: The feature cannot be enabled if the firmware does not support it.

VfCpuMonBatchPeriodSec

REG_DWORD

The frequency of issuing of automatic batch request in seconds.

Values: [0, 86400]

Default value: 60 (i.e., once in minute)

Note: The value ‘0’ means “Stop issuing automatic requests”.

VfCpuMonFwCpuUsageMax

REG_DWORD

The threshold for the FwCpuUsage counter, showing the VF CPU usage in percent.

Values: [0, 100]

Default value: 60

VfCpuMonErrors2FwMax

REG_DWORD

The threshold for ‘Errors2FW’ counter, showing the number of errors, handled by FW.

Values: [0, 0xffffffff]

Default value: 1000


© Copyright 2023, NVIDIA. Last updated on Nov 1, 2023.