image image image image image

On This Page

Mellanox IPoIB and Ethernet drivers use registry keys to control the NIC operations. The registry keys receive default values during the installation of the Mellanox adapters. Most of the parameters are visible in the registry by default, however, certain parameters must be created in order to modify the default behavior of the Mellanox driver.
The adapter can be configured either from the User Interface (Device Manager -> Mellanox Adapter -> Right click -> Properties) or by setting the registry directly.
All Mellanox adapter parameters are located in the registry under the following registry key:

HKEY_LOCAL_MACHINE
\SYSTEM
\CurrentControlSet
\ Control
\ Class
\{4D36E972-E325-11CE-BFC1-08002bE10318}
\<Index>

The registry key can be divided into 4 different groups:

GroupDescription

Basic

Contains the basic configuration.

Offload Options

Controls the offloading operation that the NIC supports.

Performance Options

Controls the NIC operation in different environments and scenarios.

Flow Control Options

Controls the TCP/IP traffic.

Any registry key that starts with an asterisk ("*") is a well-known registry key. For more details regarding the registries, please refer to:
http://msdn.microsoft.com/en-us/library/ff570865(v=VS.85).aspx

Finding the Index Value of the Network Interface

To find the index value of your Network Interface from the Device Manager please perform the following steps:

  1. Open Device Manager, and go to Network Adapters.
  2. Right click ->Properties on Mellanox Connect-X® Ethernet Adapter.
  3. Go to Details tab.
  4. Select the Driver key, and obtain the nn number.

In the below example, the index equals 0010.

All registry keys added for driver configuration should be of string type (REG_SZ).

After setting a registry key and re-loading the driver, you may use the mlx5cmd -regkeys command to assure that the value was read by the driver.

Basic Registry Keys

This group contains the registry keys that control the basic operations of the NIC 

Value NameDefault ValueDescription

*JumboPacket

ETH: 1514

IPoIB: 4092

The maximum size of a frame (or a packet) that can be sent over the wire. This is also known as the maximum transmission unit (MTU). The MTU may have a significant impact on the network's performance as a large packet can cause high latency. However, it can also reduce the CPU utilization and improve the wire efficiency. The standard Ethernet frame size is 1514 bytes, but Mellanox drivers support wide range of packet sizes.

The valid values are:

  • Ethernet: 600 up to 9600
  • IPoIB: 256 up to 4092

Note: All the devices across the network (switches and routers) should support the same frame size. Be aware that different network devices calculate the frame size differently. Some devices include the header, i.e. information in the frame size, while others do not.

Mellanox adapters do not include Ethernet header information in the frame size. (i.e when setting *JumboPacket to 1500, the actual frame size is 1514).

*ReceiveBuffers

512

The number of packets each ring receives. This parameter affects the memory consumption and the performance. Increasing this value can enhance receive performance, but also consumes more system memory.

In case of lack of received buffers (dropped packets or out of order received packets), you can increase the number of received buffers.

The valid values are 256 up to 4096.

*TransmitBuffers

2048

The number of packets each ring sends. Increasing this value can enhance transmission performance, but also consumes system memory.

The valid values are 256 up to 4096.

*NetworkDirect

1

The *NetworkDirect keyword determines whether the miniport driver's NDK functionality can be enabled. If this keyword value is set to 1 ("Enabled"), NDK functionality can be enabled. If it is set to 0 ("Disabled"), NDK functionality cannot be enabled.

 Note: This key is enabled by default, thus NDK will be used. It is important to set the switch to enable ECN and/or PFC otherwise the system will experience performance degradation.

 Note: This key affects NDK functionality and not Userspace ND (Network Direct).

 For further details, see: https://msdn.microsoft.com/en-us/windows/hardware/drivers/ network/enabling-and-disabling-ndk-functionality

*NetworkDirectTechnology

0

The *NetworkDirectTechnology keyword determines the technology used for the device.

The valid values are:

  • 0 - Device Default
  • 3 - RoCE
  • 4 - RoCE v2

For further details, see: https://docs.microsoft.com/en-us/windows-hardware/drivers/network/inf-requirements-for-ndkpi

General Registry Keys


Key NameKey TypeValuesDescription

ThreadedDpcEnable

DWORD

  • 0 - Disabled
  • 1 - Enabled

Enabling Threaded DPC. See Threaded DPC.

CheckForHangTOInSeconds

REG_DWORD

[0 – MAX_ULONG]

Default: 4

The interval in seconds for the Check- for-Hang mechanism

 Note: This registry key is available only when using WinOF-2 v2.0 and later.

 Note: As of WinOF-2 v2.10, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.


Offload Registry Keys

This group of registry keys allows the administrator to specify which TCP/IP offload settings are handled by the adapter rather than by the operating system.

Enabling offloading services increases transmission performance. Due to offload tasks (such as checksum calculations) performed by adapter hardware rather than by the operating system (and, therefore, with lower latency). In addition, CPU resources become more available for other tasks. 

Value NameDefault ValueDescription

*LsoV2IPv4

1

Large Send Offload Version 2 (IPv4). The valid values are:

  • 0: disable
  • 1: enable

*LsoV2IPv6

1

Large Send Offload Version 2 (IPv6). The valid values are:

  • 0: disable
  • 1: enable

LSOSize

64000

The maximum number of bytes that the TCP/IP stack can pass to an adapter in a single packet.

This value affects the memory consumption and the NIC performance. The valid values are MTU+1024 up to 64000.

Note: This registry key is not exposed to the user via the UI. If LSOSize is smaller than MTU+1024, LSO will be disabled.

LSOMinSegment

2

The minimum number of segments that a large TCP packet must be divisible by, before the transport can offload it to a NIC for segmentation. The valid values are 2 up to 32.

Note: This registry key is not exposed to the user via the UI.

LSOTcpOptions

1

Enables that the miniport driver to segment a large TCP packet whose TCP header contains TCP options.

The valid values are:

  • 0: disable
  • 1: enable

Note: This registry key is not exposed to the user via the UI.

LSOIpOptions

1

Enables its NIC to segment a large TCP packet whose IP header contains IP options.

The valid values are:

  • 0: disable
  • 1: enable

Note: This registry key is not exposed to the user via the UI.

*IPChecksumOffloadIPv4

3

Specifies whether the device performs the calculation of IPv4 checksums. The valid values are:

  • 0: (disable)
  • 1: (Tx Enable)
  • 2: (Rx Enable)
  • 3: (Tx and Rx enable)

*TCPUDPChecksumOffl oadIPv4

3

Specifies whether the device performs the calculation of TCP or UDP checksum over IPv4.

The valid values are:

  • 0: (disable)
  • 1: (Tx Enable)
  • 2: (Rx Enable)
  • 3: (Tx and Rx enable)

*TCPUDPChecksumOffl oadIPv6

3

Specifies whether the device performs the calculation of TCP or UDP checksum over IPv6.

The valid values are:

  • 0: (disable)
  • 1: (Tx Enable)
  • 2: (Rx Enable)
  • 3: (Tx and Rx enable)

*RssOnHostVPorts

1

Virtual Machine Multiple Queue (VMMQ) HW Offload The valid values are:

  • 0: disable
  • 1: enable

SwParsing

N/A

Specifies whether the device performs the calculation of TCP checksum over IP-in-IP encapsulated IPv4/6 sent packets.

The valid values are:

  • 0: (disable)
  • 1: (Tx Enable)

Performance Registry Keys

This group of registry keys configures parameters that can improve adapter performance.

Value NameDefault ValueDescription

TxIntModerationProfile

1

Enables the assignment of different interrupt moderation profiles for send completions. Interrupt moderation can have great effect on optimizing network throughput and CPU utilization.

The valid values are:

  • 0: Low Latency
    Implies higher rate of interrupts to achieve better latency, or to handle scenarios where only a small number of streams are used.
  • 1: Moderate
    Interrupt moderation is set to midrange defaults to allow maximum throughput at minimum CPU utilization for common scenarios.
  • 2: Aggressive
    Interrupt moderation is set to maximal values to allow maximum throughput at minimum CPU utilization for more intensive, multi-stream scenarios.
  • 3: Dynamic
    Improve existing system performance by changing interrupt moderation dynamically while also decreasing latency and CPU usage

Note: As of WinOF-2 v2.10, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

RxIntModerationProfile

1

Enables the assignment of different interrupt moderation profiles for receive completions. Interrupt moderation can have a great effect on optimizing network throughput and CPU utilization.

The valid values are:

  • 0: Low Latency
    Implies higher rate of interrupts to achieve better latency, or to handle scenarios where only a small number of streams are used.
  • 1: Moderate
    Interrupt moderation is set to midrange defaults to allow maximum throughput at minimum CPU utilization for common scenarios.
  • 2: Aggressive
    Interrupt moderation is set to maximal values to allow maximum throughput at minimum CPU utilization, for more intensive, multi-stream scenarios.
  • 3: Dynamic
    Improve existing system performance by changing interrupt moderation dynamically while also decreasing latency and CPU usage

Note: As of WinOF-2 v2.10, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

RecvCompletionMethod

1

Sets the completion methods of the receive packets, and it affects network throughput and CPU utilization.

The supported methods are:

  • Polling - increases the CPU utilization, because the system polls the received rings for incoming packets; however, it may increase the network bandwidth since the incoming packet is handled faster.
  • Adaptive - combines the interrupt and polling methods dynamically, depending on traffic type and network usage.

The valid values are:

  • 0: polling
  • 1: adaptive

*InterruptModeration

1

Sets the rate at which the controller moderates or delays the generation of interrupts, making it possible to optimize network throughput and CPU utilization. When disabled, the interrupt moderation of the system generates an interrupt when the packet is received. In this mode, the CPU utilization is increased at higher data rates, because the system must handle a larger number of interrupts. However, the latency is decreased, since that packet is processed more quickly.

When interrupt moderation is enabled, the system accumulates interrupts and sends a single interrupt rather than a series of interrupts.

The valid values are:

  • 0: disable
  • 1: enable

Note: As of WinOF-2 v2.10, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

RxIntModeration

2

Sets the rate at which the controller moderates or delays the generation of interrupts, making it possible to optimize network throughput and CPU utilization. The default setting (Adaptive) adjusts the interrupt rates dynamically, depending on traffic type and network usage. Choosing a different setting may improve network and system performance in certain configurations.

The valid values are:

  • 1: static
  • 2: adaptive

The interrupt moderation count and time are configured dynamically, based on traffic types and rate.

TxIntModeration

2

Sets the rate at which the controller moderates or delays the generation of interrupts, making it possible to optimize network throughput and CPU utilization. The default setting (Adaptive) adjusts the interrupt rates dynamically, depending on traffic type and network usage. Choosing a different setting may improve network and system performance in certain configurations.

The valid values are:

  • 1: static
  • 2: adaptive

The interrupt moderation count and time are configured dynamically, based on traffic types and rate.

*RSS

1

Sets the driver to use Receive Side Scaling (RSS) mode to improve the performance of handling incoming packets. This mode allows the adapter port to utilize the multiple CPUs in a multi-core system for receiving incoming packets and steering them to their destination. RSS can significantly improve the number of transactions per second, the number of connections per second, and the network throughput.

This parameter can be set to one of two values:

  • 1: enable (default)
    Sets RSS Mode.
  • 0: disable
    The hardware is configured once to use the Toeplitz hash function and the indirection table is never changed.

ThreadPoll

3000

The number of cycles that should be passed without receiving any packet before the polling mechanism stops when using polling completion method for receiving. Afterwards, receiving new packets will generate an interrupt that reschedules the polling mechanism.

The valid values are 0 up to 200000.

Note: This registry value is not exposed via the UI.

Note: As of WinOF-2 v2.10, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

VlanId

ETH: 0

Enables packets with VlanId. It is used when no team intermediate driver is used.

The valid values are:

  • 0: disable
    No VLAN Id is passed.
  • 1-4095
    Valid VLAN ID that will be passed.

Note: This registry value is only valid for Ethernet.

*NumRSSQueues

8

The maximum number of the RSS queues that the device should use.

Note: This registry key is only in Windows Server 2012 and above.

BlueFlame

1

The latency-critical Send WQEs to the device. When a BlueFlame is used, the WQEs are written directly to the PCI BAR of the device (in addition to memory), so that the device may handle them without having to access memory, thus shortening the execution latency. For best performance, it is recommended to use the BlueFlame when the HCA is lightly loaded. For high- bandwidth scenarios, it is recommended to use regular posting (without BlueFlame).

The valid values are:

  • 0: disable
  • 1: enable

Note: This registry value is not exposed via the UI.

*MaxRSSProcessors

8

The maximum number of RSS processors.

Note: This registry key is only in Windows Server 2012 and above.

AsyncReceiveIndicate



0

Disabled default

1

Enables packet burst buffering using threaded DPC

2

Enables packet burst buffering using polling

RfdReservationFactor

150

Controls the number of reserved receive packets,

Ethernet Registry Keys

The following section describes the registry keys that are only relevant to Ethernet driver.

Value NameDefault ValueDescription

RoceMaxFrameSize

1024

The maximum size of a frame (or a packet) that can be sent by the RoCE protocol (a.k.a Maximum Transmission Unit (MTU).

Using larger RoCE MTU will improve the performance; however, one must ensure that the entire system, including switches, supports the defined MTU.

Ethernet packet uses the general MTU value, whereas the RoCE packet uses the RoCE MTU

The valid values are:

  • 256
  • 512
  • 1024
  • 2048

 Note: This registry key is supported only in Ethernet drivers.

*PriorityVLANTag

3
(Packet Priority & VLAN Enabled)

Enables sending and receiving IEEE 802.3ac tagged frames, which include:

  • 802.1p QoS (Quality of Service) tags for priority-tagged packets.
  • 802.1Q tags for VLANs.

When this feature is enabled, the Mellanox driver supports sending and receiving a packet with VLAN and QoS tag.

Note: The default value on this key on the VF is 1 (Priority Enabled).

DeviceRxStallTimeout

1000

The maximum period for a single received packet processing. If the packet was not processed during this time, the device will be declared as stalled and will increase the "Critical Stall Watermark Reached" counter. The value is given in mSec. The maximum period is 8000 mSec. The special value of 0, indicates that the DeviceRxStallTimeout is active.

Range: 0x0050 (80)- 0x1F40 (8000)

DeviceRxStallWatermark

0

The maximum period for a single received packet processing. If the packet was not processed during this time, the device will increase a diagnostic counter called "Minor Stall Watermark Reached”. The value is given in mSec. The maximum period is 8000 mSec. The special value of 0 indicates that the DeviceRxStallWatermark is active

Range: 0x0050 (80)- 0x1F40 (8000)

Note: As of WinOF-2 v2.20, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

TCHeadOfQueueLifeTim eLimit

0-20

Default: 19

The time a packet can live at the head of a TC queue before it is discarded. The timeout value is defined by 4,096us multiplied by 2^TCHeadOfQueueLifetimeLimit.

TCStallCount

1-7

0: Disabled

The number of sequential packets dropped due to Head Of Queue Lifetime Limit, in order for the port to enter the TCStalled state. All packets for the TC are discarded in this state for a period of 8 times the timeout defined by TCHeadOfQueueLifetimeLimit.

TCStallEnable

0- Disabled

1- Enabled (Default)

Enables the TCStalled state.

TCHeadOfQueueLifeTim eLimitEnable0

The TCs for which Head Of Queue Lifetime Limit is enabled. Bit 0 represents TC0, bit 1 represents TC1 and so on.
The valid values are:

  • 0-255
  • 0: disabled

Flow Control Options

This group of registry keys allows the administrator to control the TCP/IP traffic by pausing frame transmitting and/or receiving operations. By enabling the Flow Control mechanism, the adapters can overcome any TCP/IP issues and eliminate the risk of data loss.

Value NameDefault ValueDescription

*FlowControl

3

When Rx Pause is enabled, the receiving adapter generates a flow control frame when its received queue reaches a pre-defined limit. The flow control frame is sent to the sending adapter.

When TX Pause is enabled, the sending adapter pauses the transmission if it receives a flow control frame from a link partner.

The valid values are:

  • 0: Flow control is disabled
  • 1: Tx Flow control is Enabled
  • 2: Rx Flow control is enabled
  • 3: Rx & Tx Flow control is enabled

DeviceRxStallTimeout

1000 mSec

When the device is in stall state (congestion mode), after the configured period of having the device in such state expires (the maximum period is 8 sec), the device will disable the Flow Control mechanism.

The valid values are:

  • Minimum: 0
  • Maximum: 8000

DeviceRxStallWatermark

0 mSec

When the device is in "stall state" (congestion mode), after the configured period of having the device in such state expires (the maximum period is 8 sec), the device will declare the driver as stalled.

The valid values are:

  • Minimum: 0
  • Maximum: 8000

VMQ Options

This section describes the registry keys that are used to control the NDIS Virtual Machine Queue (VMQ). VMQ is supported by WinOF-2 and allows a performance boost for Hyper-V VMs.

For more details about VMQ please refer to Microsoft web site, http://msdn.microsoft.com/en-us/library/windows/hardware/ff571034(v=vs.85).aspx

Value NameDefault ValueDescription

*VMQ

1

The support for the virtual machine queue (VMQ) features of the network adapter.

The valid values are:

  • 1: enable
  • 0: disable

*RssOrVmqPreference

0

Specifies whether VMQ capabilities should be enabled instead of receive- side scaling (RSS) capabilities.

The valid values are:

  • 0: Report RSS capabilities
  • 1: Report VMQ capabilities

 Note: This registry value is not exposed via the UI.

*VMQVlanFiltering

1

Specifies whether the device enables or disables the ability to filter network packets by using the VLAN identifier in the media access control (MAC) header.

The valid values are:

  • 0: disable
  • 1: enable

RoCE Options

This section describes the registry keys that are used to control RoCE mode.

Value NameDefault ValueDescription

roce_mode

0 - RoCE

The RoCE mode. The valid values are:

  • 0 - RoCE
  • 4 - No RoCE

 Note: The default value depends on the WinOF package used.

SR-IOV Options

This section describes the registry keys that are used to control the NDIS Single Root I/O Virtualization (SR-IOV). The SR-IOV is supported by WinOF-2 and allows a performance boost for Hyper-V VMs.

For more details about the VMQ, please see Single Root I/O Virtualization (SR-IOV) on Microsoft website.

Value NameDefault ValueDescription

*SRIOV

1

The support for the SR-IOV features of the network adapter. The valid values are:

  • 1: enable
  • 0: disable

*SriovPreferred

N/A (hidden)

A value that defines whether SR-IOV capabilities should be enabled instead of the virtual machine queue (VMQ), or receive side scaling (RSS) capabilities.

MaxFWPagesUsagePerVF

250000

This key sets the limitation for the maximum number of 4KB pages that the host could allocate for VFs resources. When set to 0, limitation is disabled.

RDMA Registry Keys

The following section describes the registry keys that are only relevant to RDMA.

Value NameDefault ValueDescription

EnableguestRdma

1: Enabled

Able to prevent RDMA in the VF from the host. This feature is enabled by default.

Note: This registry key is applicable to Ethernet networks only.

MaxCMRetries

15

Maximum number of times that either party can re-send a REQ, REP, or

DREQ message. After re-sending for the maximum number of times without a response, the sending party should then terminate the protocol

by sending a REJ message indicating that it timed out.

RemoteCMResponseTimeout

16

Expressed as 4.096 microSec * 2 ^ cm_response_timewait, within which the CM message recipient shall transmit a response to the sender.

Valid values are: 3-25

NetworkDirectAdminOnly

0

In case this key is set 1, only an Admin user can use the ND - NetworkDirect application.

Max value: 1

This registry key can be found at: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\ mlx5\Parameters

Diagnostics Registry Keys

Dump Me Now (DMN) Registry Keys

The registry keys for the DMN feature are located at: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\{4d36e972-e325-11ce- bfc1-08002be10318}\nnnn\

For further information on how to find the registry keys, refer to section Finding the Index Value of the Network Interface.

The following section describes the registry keys that configure the Dump Me Now feature (see section Dump Me Now (DMN)).

Value NameKey TypeDescription

DumpMeNowDirectory

REG_SZ

Path to the root directory in which the DMN places its dumps. The path should be provided in a kernel path style, which means prefixing the drive name with "\??\" (e.g. \??\C:\DMN_DIR). BDF will be added to specified name. (e.g. if specified directory name is \??\C:\DMN_DIR, then directory \??\C:\DMN_DIR-<b>-<d>-<f> will be created for Host and \??\C:\DMN_DIR-<b>-<d> for VF)

Default Value:

  • Host: \Systemroot\temp\Mlx5_Dump_Me_Now-<b>-<d>-<f>
  • VF: \Systemroot\temp\Mlx5_Dump_Me_Now-<b>-<d>-0

DumpMeNowTotalCount

REG_DWORD

The maximum number of allowed DMN dumps. Newer dumps beyond this number will override old ones.

Values: [0,512]

Default Value: 128

Note: As of WinOF-2 v2.10, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

DumpMeNowPreservedCount

REG_DWORD

Specifies the number of DMN dumps that will be reserved, and will never be overridden by newer DMN dump.

Values: [0,512]

Default Value: 8

Note: As of WinOF-2 v2.10, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

Setting DumpMeNowTotalCount and DumpMeNowPreservedCount to "0" will disable the DMN feature.

FwTrace Registry Keys

The following section describes the registry keys that configure the FwTrace feature (see section FwTrace).

Value NameKey TypeDescription

FwTracerEnabled

REG_DWORD

  • 0 - FwTrace is disabled
  • 1 - FwTrace is enabled

Values: [0,1]

Default Value: 1

Note: As of WinOF-2 v2.10, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.

FwTracerBufferSize

REG_DWORD

FwTracer Buffer Size in Bytes. This value is rounded up to be equal to 2^N * 4096 bytes.

Values: [0x2000, 0x200000]

Default Value: 0x10000

Note: As of WinOF-2 v2.10, this key can be changed dynamically. In any case of an illegal input, the value will fall back to the default value and not to the last value used.