Kubernetes Performance Tuning

This section provides a configuration example for Kubernetes performance tuning for SR-IOV.

The machine in this example includes the following CPUs:

Copy
Copied!

            
            numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 12 13 14 15 16 17
node 0 size: 31990 MB
node 0 free: 25314 MB
node 1 cpus: 6 7 8 9 10 11 18 19 20 21 22 23
node 1 size: 32237 MB
node 1 free: 27135 MB
node distances:
node 0 1 
0: 10 21 
1: 21 10

3 CPUs are reserved on NUMA, and 1 for the system and for Kubernetes. Edit /var/lib/kubelet/config.yaml:

Copy
Copied!

            
            cpuManagerPolicy: static
reservedSystemCPUs: 6-8
topologyManagerPolicy: single-numa-node

Using isolcpus kernel boot command-line isolates the CPUs from the kernel scheduler. This will ensure that a user-space process will not be scheduled by the kernel. In this example, CPUs 0-5 and 9-23 are isolated. isolcpus=0,1,2,3,4,5,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24.

Warning

In case of changes in the reservedSystemCPUs or in the cpuManagerPolicy the /var/lib/kubelet/cpu_manager_state should be deleted, and the kubelet should be restarted.