NVIDIA MLNX_OFED Documentation v5.9-0.5.6.0.107
Linux Kernel Upstream Release Notes v5.17

RSS Hash Function

The device has the ability to use XOR as the RSS distribution function, instead of the default Toplitz function.
The XOR function can be better distributed among driver's receive queues in a small number of streams, where it distributes each TCP/UDP stream to a different queue.MLNX_OFEDMLNX_EN provides the following option to change the working RSS hash function from Toplitz to XOR, and vice-versa:

Through sysfs, located at: /sys/class/net/eth*/settings/hfunc.

Procedure_Heading_Icon-version-1-modificationdate-1701078755244-api-v2.PNG

To query the operational and supported hash functions:

Copy
Copied!
            

cat /sys/class/net/eth*/settings/hfunc

Example:

Copy
Copied!
            

cat /sys/class/net/eth2/settings/hfunc Operational hfunc: toeplitz Supported hfuncs: xor toeplitz

To change the operational hash function:

Procedure_Heading_Icon-version-1-modificationdate-1701078755244-api-v2.PNG

Copy
Copied!
            

echo xor > /sys/class/net/eth*/settings/hfunc

Receive Side Scaling (RSS) technology allows spreading incoming traffic between different receive descriptor queues. Assigning each queue to different CPU cores allows better load balancing of the incoming traffic and improves performance.
This technology was extended to user space by the verbs layer and can be used for RAW ETH QP.

Steering rules classify incoming packets and deliver a specific traffic type (e.g. TCP/UDP, IP only) or a specific flow to "RX Hash" QP. "RX Hash" QP is responsible for spreading the traffic it handles between the Receive Work Queues using RX hash and Indirection Table. The Receive Work Queue can point to different CQs that can be associated with different CPU cores.

The below verbs should be used to achieve this task in both control and data path. Details per verb should be referenced from its man page.

  • ibv_create_wq, ibv_modify_wq, ibv_destory_wq

  • ibv_create_rwq_ind_table, ibv_destroy_rwq_ind_table

  • ibv_create_qp_ex with specific RX configuration to create the "RX hash" QP

© Copyright 2023, NVIDIA. Last updated on Nov 27, 2023.