This section provides a configuration example for Kubernetes performance tuning for SR-IOV.
The machine in this example includes the following CPUs:
numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 12 13 14 15 16 17
node 0 size: 31990 MB
node 0 free: 25314 MB
node 1 cpus: 6 7 8 9 10 11 18 19 20 21 22 23
node 1 size: 32237 MB
node 1 free: 27135 MB
node distances:
node 0 1
0: 10 21
1: 21 10
3 CPUs are reserved on NUMA, and 1 for the system and for Kubernetes. Edit /var/lib/kubelet/config.yaml:
cpuManagerPolicy: static
reservedSystemCPUs: 6-8
topologyManagerPolicy: single-numa-node
Using isolcpus kernel boot command-line isolates the CPUs from the kernel scheduler. This will ensure that a user-space process will not be scheduled by the kernel. In this example, CPUs 0-5 and 9-23 are isolated. isolcpus=0,1,2,3,4,5,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24.
In case of changes in the reservedSystemCPUs or in the cpuManagerPolicy the /var/lib/kubelet/cpu_manager_state should be deleted, and the kubelet should be restarted.