Network Operator Application Notes 23.10.0 - Sphinx Test
1.0

Create Kubernetes cluster with Kubespray

For an High Availability control plane, 3 hosts are required with the following requirements:

  • 16G RAM

  • 100 GB DISK

  • 6 CPU cores

First, provision the hosts OS with Ubuntu 22.04.

For the installation of Kubernetes, Kubespray can be used. See this link for extended instructions.

Since Kubespray is using Ansible an additional machine is needed to run it.

It is recommended to copy the machine’s public SSH key to the hosts with ssh-copy-id.

Read the following section regarding the needed libraries for Ansible and other requirements.

Clone the Kubespray GitHub repository.

Copy
Copied!
            

git clone https://github.com/kubernetes-sigs/kubespray.git cd kubespray git checkout release-2.20

Create an inventory file for Ansible. (See more details here).

In the following example, the HOST_PREFIX will set the nodes name prefix to dcmp-cp- and KUBE_CONTROL_HOSTS will mark all 3 nodes to be control nodes.

Copy
Copied!
            

cp -r inventory/sample inventory/mycluster declare -a IPS=(10.10.1.3 10.10.1.4 10.10.1.5) HOST_PREFIX=dcm-cp- KUBE_CONTROL_HOSTS=3 CONFIG_FILE=inventory/mycluster/hosts.yaml python3 contrib/inventory_builder/inventory.py ${IPS[@]}

Customize following variables in inventory/mycluster/group_vars/k8s_cluster/k8s-cluster.yml:

Copy
Copied!
            

# Make a copy of kubeconfig on the host that runs Ansible in {{ inventory_dir }}/artifacts kubeconfig_localhost: true kube_network_plugin: flannel

For OVN-Kubernetes, specify kube_network_plugin: cni to prevent Kubespray from installing any CNI. Otherwise, use kube_network_plugin: flannel

For high availability , configuring a VIP and load balancer is required. See instructions.

Here a sample configuration:

In inventory/mycluster/group_vars/k8s_cluster/k8s-cluster.yml, set:

Copy
Copied!
            

kube_proxy_strict_arp: true

In inventory/mycluster/group_vars/all/all.yml, set: (Make sure to update values with correct VIP and interface)

Copy
Copied!
            

kube_vip_arp_enabled: true kube_vip_enabled: true kube_vip_controlplane_enabled: true kube_vip_address: 10.7.12.250 # update with correct VIP kube_vip_interface: eno1 # update with correct interface kube_vip_services_enabled: false loadbalancer_apiserver: address: "{{ kube_vip_address }}" port: 6443

Copy
Copied!
            

sudo ansible-playbook --flush-cache -T 30 -i inventory/mycluster/hosts.yaml cluster.yml --become --become-user=root --private-key=~/.ssh/private_key

A copy of the Kubeconfig file will be available in inventory/mycluster/artifacts/admin.conf.

Note

In case of kube_network_plugin: flannel, due to an issue in Kubespray, the following comand should be executed:

Copy
Copied!
            

kubectl -n kube-system patch ds kube-flannel-ds-arm64 --type "json" -p '[{"op":"add","path":"/spec/template/spec/initContainers/0/image","value":"docker.io/flannelcni/flannel-cni-plugin:v1.1.0-arm64"}]'

Previous Vault PKI configuration
Next DPU settings
© Copyright 2023, NVIDIA. Last updated on Feb 7, 2024.