Multi-PMD Configuration

Poll Mode Driver (PMD) threads are responsible for performing datapath processing of packets that traverse the software datapath (e.g., slow path) and for handling connection tracking (CT) offloads.

Scaling PMD threads can improve those on systems with multiple cores. The following configuration steps enable you to control how PMD threads are scheduled and how ports are assigned to RX queues.

Set the pmd-cpu-mask in other_config to specify which CPU cores are used by PMD threads. By default, Open vSwitch (OVS) assigns one core per NUMA node.

For ESW manager ports, configure multiple RX queues to allow parallel packet processing.

The following is an example command to configure N RX queues for port P:

            

            
ovs-vsctl set interface <P> options:n_rxq=<N>
        

OVS automatically schedules port-queue tuples across available PMD threads, using only cores from the card’s NUMA node by default.

You can customize this scheduling by setting the pmd-rxq-affinity on the interface, using <QUEUE>:<CORE> tuples:

            

            
ovs-vsctl set interface <P> other_config:pmd-rxq-affinity="0:4,1:17"
        

In this example:

• Queue 0 is scheduled on core 4

• Queue 1 is scheduled on core 17

To view the current PMD thread-to-queue scheduling, run:

            

            
ovs-appctl dpif-doca/pmd-rxq-show
        

CT offload size is configured by hw-offload-ct-size (500,000 by default). This size is distributed among the number of PMDs, configured by pmd-cpu-mask ) (default is by the number of NUMAs in the system).

Each port accesses to CT according to number of RX queue it use (n_rxq).

For example if n_rxq is set to 2 but pmd-cpu-mask has 3 PMDs, only 2/3 of the doca-ct size can be utilized.

To overcome it, either align n_rxq to the number of PMDs if applicable (single NUMA systems) or increase hw-offload-ct-size by this factor (3/2 in this case).