Create Content



Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Created on Jan 20, 2020

Introduction

This document describes how to install Kubernetes(K8s) cluster with kubespray.

This guide assumes the following:

  • Hardware
    All servers used during deployment have the same hardware specification and equipped with a Mellanox ConnectX network card.
  • Software 
    Ubuntu Server 18.04 operating system is installed on all servers with OpenSSH server packages.
  • Network 
    DHCP server and DNS server is part of the IT infrastructure.
    For each server, the server name and IP address are reserved on the DHCP server.
    All servers used in our deployment are connected to Ethernet switch by single 100GbE Ethernet copper cable and have Internet access.


Info

This deployment is HW independent and we use Mellanox gears as an example.

References

Solution Overview

Equipment

For K8s cluster deployment must use the same type of servers.
For additional info please refer to Kubespray Requirements.


Solution Logical Design

The MGMT node is only used to deploy a K8s cluster through Node1-Node5.


Image Modified

Network Configuration

Server names with network configurations are provided in the below table: 


Host Configuration

Ubuntu Server 18.04 operating system is installed on all servers with OpenSSH server packages.
Please update/upgrade Ubuntu software packages and install the latest HWE kernel by running the below commands:

Code Block
languagetext
themeMidnight
titleServer console
# apt-get update
# apt-get -y install linux-image-generic-hwe-18.04
# reboot

Deployment Guide

K8s Cluster Deployment and Configuration

SSH Private Key and SSH Passwordless Login

Please login to the Management server as root user and create SSH private key for configuring the password-less authentication on your computer by running the following command:

Code Block
languagetext
themeMidnight
# ssh-keygen

Copy your SSH private key, such as ~/.ssh/id_rsa, to all nodes in your deployment by running the following command:

Code Block
languagetext
themeMidnight
# ssh-copy-id root@nodename

Kubespray Configuration

Please install dependencies for running Kubespray with Ansible on the Management server:

Code Block
languagetext
themeMidnight
# cd ~
# apt -y install python3-pip
# git clone https://github.com/kubernetes-sigs/kubespray.git
# cd kubespray
# pip3 install -r requirements.txt

Create new cluster configuration:

Code Block
languagetext
themeMidnight
# cp -rfp inventory/sample inventory/mycluster
# declare -a IPS=(192.168.222.111 192.168.222.101 192.168.222.102 192.168.222.103 192.168.222.104)
# CONFIG_FILE=inventory/mycluster/hosts.yaml python3 contrib/inventory_builder/inventory.py ${IPS[@]}

Review and change the host configuration file - inventory/mycluster/hosts.yaml.

Example:

Code Block
languagetext
themeMidnight
titleinventory/mycluster/hosts.yaml
all:
  hosts:
    node1:
      ansible_host: 192.168.222.111
      ip: 192.168.222.111
      access_ip: 192.168.222.111
    node2:
      ansible_host: 192.168.222.101
      ip: 192.168.222.101
      access_ip: 192.168.222.101
    node3:
      ansible_host: 192.168.222.102
      ip: 192.168.222.102
      access_ip: 192.168.222.102
    node4:
      ansible_host: 192.168.222.103
      ip: 192.168.222.103
      access_ip: 192.168.222.103
    node5:
      ansible_host: 192.168.222.104
      ip: 192.168.222.104
      access_ip: 192.168.222.104
  children:
    kube-master:
      hosts:
        node1:
    kube-node:
      hosts:
        node2:
        node3:
        node4:
        node5:
    etcd:
      hosts:
        node1:
    k8s-cluster:
      children:
        kube-master:
        kube-node:
    calico-rr:
      hosts: 

Review and change cluster installation parameters under inventory/mycluster/group_vars:

Code Block
languagetext
themeMidnight
# cat inventory/mycluster/group_vars/all/all.yml
# cat inventory/mycluster/group_vars/k8s-cluster/k8s-cluster.yml
Info

The default Kubernetes CNI can be changed by setting the desired kube_network_plugin valueparameter in inventory/mycluster/group_vars/k8s-cluster/k8s-cluster.yml.

Install K8s Cluster Using Ansible Playbook

Deploy K8s cluster with Kubespray Ansible Playbook:

Code Block
languagetext
themeMidnight
# ansible-playbook -i inventory/mycluster/hosts.yaml --become --become-user=root cluster.yml

The execution time for this step may take a while to finalize.

Example of a successful completion of the playbooks:

Code Block
languagetext
themeMidnight
collapsetrue
...
PLAY RECAP *********************************************************************************************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0   
node1                      : ok=611  changed=137  unreachable=0    failed=0   
node2                      : ok=442  changed=92   unreachable=0    failed=0   
node3                      : ok=404  changed=88   unreachable=0    failed=0   
node4                      : ok=404  changed=88   unreachable=0    failed=0   
node5                      : ok=404  changed=88   unreachable=0    failed=0   

Wednesday 15 January 2020  10:39:14 +0200 (0:00:00.164)       0:16:20.903 ***** 
=============================================================================== 
container-engine/docker : ensure docker packages are installed --------------------------------------------------------------------------------------------------------------------- 65.67s
download : download_file | Download item ------------------------------------------------------------------------------------------------------------------------------------------- 53.43s
download : download_file | Download item ------------------------------------------------------------------------------------------------------------------------------------------- 50.71s
kubernetes/master : kubeadm | Initialize first master ------------------------------------------------------------------------------------------------------------------------------ 45.96s
kubernetes/kubeadm : Join to cluster ----------------------------------------------------------------------------------------------------------------------------------------------- 42.65s
download : download_file | Download item ------------------------------------------------------------------------------------------------------------------------------------------- 28.74s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 27.37s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 22.59s
kubernetes/master : Master | wait for kube-scheduler ------------------------------------------------------------------------------------------------------------------------------- 22.41s
bootstrap-os : Install python ------------------------------------------------------------------------------------------------------------------------------------------------------ 18.72s
kubernetes/preinstall : Install packages requirements ------------------------------------------------------------------------------------------------------------------------------ 15.50s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 14.98s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 12.85s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 12.33s
download : download_container | Download image if required ------------------------------------------------------------------------------------------------------------------------- 12.08s
download : download_container | Download image if required -------------------------------------------------------------------------------------------------------------------------- 9.86s
download : download_file | Download item -------------------------------------------------------------------------------------------------------------------------------------------- 9.61s
etcd : wait for etcd up ------------------------------------------------------------------------------------------------------------------------------------------------------------- 8.91s
container-engine/docker : ensure docker-ce repository is enabled -------------------------------------------------------------------------------------------------------------------- 8.81s
bootstrap-os : Install dbus for the hostname module --------------------------------------------------------------------------------------------------------------------------------- 7.98s


Deployment Verification

Verify that the Kubernetes cluster is running by executing the following command on the K8s Master Node:

Code Block
languagetext
themeMidnight
root@node1:~# kubectl get nodes -o wide
NAME    STATUS   ROLES    AGE    VERSION   INTERNAL-IP       EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
node1   Ready    master   4h   v1.16.3   192.168.222.111   <none>        Ubuntu 18.04.3 LTS   5.0.0-37-generic   docker://18.9.7
node2   Ready    <none>   4h   v1.16.3   192.168.222.101   <none>        Ubuntu 18.04.3 LTS   5.0.0-37-generic   docker://18.9.7
node3   Ready    <none>   4h   v1.16.3   192.168.222.102   <none>        Ubuntu 18.04.3 LTS   5.0.0-37-generic   docker://18.9.7
node4   Ready    <none>   4h   v1.16.3   192.168.222.103   <none>        Ubuntu 18.04.3 LTS   5.0.0-37-generic   docker://18.9.7
node5   Ready    <none>   4h   v1.16.3   192.168.222.104   <none>        Ubuntu 18.04.3 LTS   5.0.0-37-generic   docker://18.9.7



Done!

Related Documents

Content by Label
showLabelsfalse
showSpacefalse
sortcreation
cqllabel = "k8s"