Overview

Kubernetes is an open source container orchestration engine for automating deployment, scaling and management of containerized applications. This document aims to guide Kubernetes administrators/users on the NVIDIA networking stack, offering solutions, and explaining how to configure them in Kubernetes, as well as providing detailed information on how to configure CNIs, Device Plugins and NVIDIA Network Operator with NVIDIA hardware.

k8s-images repository is provided with Dockerfile examples of how to build RDMA/DPDK/perftest container images.

Multi-network POD

By default, Kubernetes allows for a single network (primary network) to be connected to a POD. Kubernetes network attachment definition custom resources enhance this capability, and allow users to attach multi-networks for POD, a primary network which runs all of Kubernetes services and one or more secondary networks which are typically used for high performance. The cluster network CNI plugin (primary plugin) is satisfying Kubernetes’ networking requirements. Below is a list of well known cluster network CNI providers:

The Multus CNI plugin enables attaching multiple network interfaces to pods. Multus is acting as a "meta-plugin", a CNI plugin that can call multiple other CNI plugins.

Device Plugin

Kubernetes provides a device plugin framework that can be used to advertise system hardware resources to the Kubelet. More information about the device plugin framework can be found at https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/.

This document presents configuration with the following device plugins: 

• SR-IOV network device plugin - A device plugin for discovering and advertising the SR-IOV virtual functions (VFs) that are available on a Kubernetes host.
• RDMA shared device plugin - device plugin for sharing RDMA devices between PODs on the same host.