Performance Tuning
Driver Configuration
The virtio-net driver can configure the number of combined channels via ethtool. This determines how many virtqueues (VQs) can be used for the netdev. Normally, more VQs result in better overall throughput when if multi-threaded (e.g., iperf with multiple streams).
[host]# ethtool -l eth0
Channel parameters for eth0:
Pre-set maximums:
RX: n/a
TX: n/a
Other: n/a
Combined: 31
Current hardware settings:
RX: n/a
TX: n/a
Other: n/a
Combined: 15
Therefore, it is common to pick a larger number (less than pre-set maximums) of channels using the following command.
Normally, configuring the combined number of channels to be the same as number of CPUs available on the guest OS will yield good performance.
[host]# ethtool -L eth0 combined 31
[host]# ethtool -l eth0
Channel parameters for eth0:
Pre-set maximums:
RX: n/a
TX: n/a
Other: n/a
Combined: 31
Current hardware settings:
RX: n/a
TX: n/a
Other: n/a
Combined: 31
Device Configuration
To reach the best performance, it is required to make sure each tx/rx queue has an assigned MSIX. Check the information of a particular device and make sure num_queues is less than num_msix.
[dpu]# virtnet query -p 0 -b | grep -i num_
"num_msix": "64",
"num_queues": "8",
If num_queues is greater than num_msix, it is necessary to change mlxconfig to reserve more MSIX than queues. It is determined by the VIRTIO_NET_EMULATION_NUM_VF_MSIX and VIRTIO_NET_EMULATION_NUM_MSIX. Please refer to the "Virtio-net Deployment" page for more information.
By default, queue depth is set to 256. It is common to use a larger queue depth (e.g., 1024). This cannot be requested from the driver side but must be done from the device side.
Refer to the "Virtnet CLI Commands" page to learn how to modify device max_queue_size.