ESXi 7.0: NVIDIA NATIVE Driver for VMware vSphere User Manual v4.19.71.1
Linux Kernel Upstream Release Notes v6.5

Module Parameters

To set nmlx5_core parameters:

Copy
Copied!
            

esxcli system module parameters set -m nmlx5_core -p <parameter>=<value>

To set nmlx5_rdma parameters:

Copy
Copied!
            

esxcli system module parameters set -m nmlx5_rdma -p <parameter>=<value>

To show the values of the parameters:

Copy
Copied!
            

esxcli system module parameters list -m <module name>

For the changes to take effect, reboot the host.

Name

Description

Values

DRSS

Number of hardware queues for Default Queue (DEFQ) RSS.

Note: This parameter replaces the previously used “drss” parameter which is now obsolete.

  • 2-16

  • 0 - disabled

When this value is != 0, DEFQ RSS is enabled with 1 RSS Uplink queue that manages the 'drss' hardware queues.

Notes:

  • The value must be a power of 2.

  • The value must not exceed num. of CPU cores.

  • Setting the DRSS value to 16, sets the Steering Mode to device RSS

DYN_RSS

Enables dynamic RSS.

  • 0 - disabled (default)

  • 1 - enabled

RSS must be enabled to utilize dynamic RSS. Dynamic RSS allows indirection table changes during traffic, resulting in better RX packets distribution between numerous HW RX rings

GEN_RSS

Requested max. number of Generic RSS engines.

  • 0 - disabled

  • 3 - max. rss engines, including default queue

Default: 2

Generic RSS extends dynamic RSS functionality for multiple RSS queues. Generic RSS and Dynamic RSS are mutually exclusive, if both dynamic and generic RSS module parameters are set, the driver's default is set to generic RSS

Generic RSS requires DRSS or/and RSS params:

  • RSS to specify the total amount of HW queues for NetQ RSS engines, minimum is 4 per each engine. Example: RSS=8, GEN_RSS=3 / RSS=4, GEN_RSS=2

  • DRSS to specify total amount of HW queues for default RSS queue engine, minimum is 4

Notes:

The number of engines include default RSS queue. However, the driver will eventually advertise the number of RSS engines based the hardware resources, the amount of specified secondary queues, and the minimal required for RSS engine.

During runtime, it is kernel decision to utilize various engines amount and amount of queues per each engine.

rss

Number of HW queues for NETQ RSS

Values : 2-8,

0 - disabled
When this value is != 0, NETQ RSS is enabled with 1 RSS Uplink queue that manages 'RSS' HW queues.

Notes:

  • The value must be a power of 2

  • Must not exceed num. of CPU cores

dcbx

Set DCBX operational mode

  • 0 – Disabled

  • 1 - Enabled Hardware Mode

  • 2 - Enabled Software Mode

  • 3 - If Hardware Mode is supported Enable Hardware Mode, else Enable Software Mode (Default)

ecn

Enables the ECN feature.

  • 1 - enabled

  • 0 - disabled (Default)

enable_nmlx_debug

Enables debug prints for the core module.

  • 1 - enabled

  • 0 - disabled (Default)

max_vfs

max_vfs is an array of comma separated integer values, that represent the amount of VFs to open from each port.

For example: max_vfs = 1,1,2,2, will open a single VF per port on the first NIC and 2 VFs per port on second NIC. The order of the NICs is determined by pci SBDF number.

Note: VFs creation based on the system resources limitations.

  • 0 - disabled (Default)

N number of VF to allocate over each port

Note: The amount of values provided in the max_vfs array should not exceed the supported_num_ports module parameter value.

mst_recovery

Enables recovery mode (only NMST module is loaded).

  • 1 - enabled

  • 0 - disabled (Default)

pfcrx

Priority based Flow Control policy on RX.

  • 0-255

  • 0 - default

It is an 8 bits bit mask, where each bit indicates a priority [0-7].

Bit values:

  • 1 - respect incoming PFC pause frames for the specified priority.

  • 0 - ignore incoming pause frames on the specified priority.

Note: The pfcrx and pfctx values must be identical.

pfctx

Priority based Flow Control policy on TX.

  • 0-255

  • 0 - default

It is an 8 bits bit mask, where each bit indicates a priority [0-7].

Bit values:

  • 1 - generate pause frames according to the RX buffer threshold on the specified priority.

  • 0 - never generate pause frames on the specified priority.

Note: The pfcrx and pfctx values must be identical.

RSS

Number of hardware queues for NetQ RSS.

Note: This parameter replaces the previously used “rss” parameter which is now obsolete.

  • 2-8

  • 0 - disabled

When this value is != 0, NetQ RSS is enabled with 1 RSS uplink queue that manages the 'rss' hardware queues.

Notes:

  • The value must be a power of 2

  • The maximum value must be lower than the number of CPU cores.

supported_num_ports

Sets the maximum supported ports.

1-8

Default 1

Note: Before installing new cards, you must modify the maximum number of the supported ports to include the additional new ports.

trust_state

Port policy to calculate the switch priority and packet color based on incoming packet.

  • 1 - TRUST_PCP (Default)

  • 2 - TRUST_DSCP

vf_mempool_extend

Extend the VF mempool size

Do not change unless IB/RoCE VFs run out of memory.

Note: This parameter is ignored if max_vfs=0.

X - add X MBs to the VF mempool size
Default: 0

dropless_rq

Enable dropless RQ

When this feature is enabled, packets that are received while the software ring is full will not be immediately dropped.

The firmware will wait a certain period of time assuming the software will post new WQEs, otherwise the packets will be dropped.

  • 1 - enabled

  • 0 – disabled

Default: 0

ens_fallback_model

Fallback ENS Model, in case ESXi/NSX does not support quiring ENS models, this module parameter will specify to which model the driver will fallback to.

  • 0 - Model 0

  • 1 - Model 1 level 1

Default: 1

netq_rss_ens

Enables NetQ RSS for ENS model 0, which is currently not supported by NSX/ESXi.

Note: Enabling this feature will result in PSOD. We do not recommend enabling it.

  • 1 - enabled

  • 0 – disabled

Default: 0

sriov_mc_isolation

Isolates multicast traffic for SR-IOV only.

Note: Enabling this feature may cause vmknic, emulated NICs multicast and IPv6 traffic loss.

  • 1 - enabled

  • 0 – disabled

Default: 0

Name

Description

Values

dscp_force

DSCP value to force on outgoing RoCE traffic.

  • -1 - Disabled (Default)

  • 0-63 - DSCP value to force

dscp_to_pcp

Map the three MSBs of DSCP value to PCP on outgoing RoCE traffic.

Note: Cannot be enabled when pcp_force is active.

  • 0 – Disabled (Default)

  • 1 - Enabled

enable_nmlx_debug

Enable debug prints

  • 1 - enabled

  • 0 - disabled (Default)

pcp_force

PCP value to force on outgoing RoCE traffic.

Cannot be active when dscp_to_pcp is enabled.

  • -1 - Disabled (Default)

  • 0-7 - PCP value to force

roce_version

RoCE versions enable option

  • 1 - RoCE V1

  • 2 - RoCE V2

  • 3 - RoCE V1 + RoCE V2 (Default)

© Copyright 2023, NVIDIA. Last updated on Aug 31, 2023.