Created on Jul 11, 2019
This post presents how to build a Docker container with compiled Nvidia Network DPDK PMD.
References
Docker installation reference - Install Docker | Docker Documentation
Hypervisor Prerequisite
Install Docker version >= 1.9.
Install Nvidia networks rdma-core package:
CLI Command
yum install -y rdma-core-devel
Creating Docker Container Image
Download Dockerfile from Github and build the image locally on the hypervisor.
CLI Commands
git clone https://github.com/Mellanox/mlnx_docker_dpdk
cd mlnx_docker_dpdk/
## Example for OS: CentOS-7.7.1908 DPDK Version: 20.02 MOFED Version: 5.1-2.3.7.1 ##
docker build --build-arg OS_VER=7.7.1908 --build-arg DPDK_VER=20.02 --build-arg OFED_VER=5.1-2.3.7.1 -t centos/<image_name>:<version_name> .
OS_VER - CentOS image version
DPDK_VER - DPDK version
OFED_VER - Nvidia OFED version. Not specifying MOFED version will install default MOFED upstream packages.
<image_name> and <version_name> - Can be any user defined name
<image_name> and <version_name> can be any user-defined name
Note: Default master branch supports only CentOS base OS.
ubuntu18.04, ubuntu14.04 , ubuntu16.04 , rhel7.2 , rhel7.3 can be used with different Dockerfile which is available in this git project under unique bran
Running Docker Container with Nvidia Network DPDK PMD
Define and mount hugepages on the hypervisor
Allocate 1G size of hugepages:
CLI Command
echo 4 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages
OR
Allocate 2Mb size of hugepages:
CLI Commands
echo 2048 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
Mount hugespages:
mount -t hugetlbfs -o pagesize=1G none /dev/hugepages
mkdir -p /mnt/huge_c0
mount -t hugetlbfs -o pagesize=1G none /mnt/huge_c0/
This example exhibits allocation of 4Gb total hugepages for both 2Mb and 1Gb hugepages sizes. You may define different amounts relative to the free amount of memory on your hypervisor.
Define the network for the container
The recommended type of network attachment is SRIOV.
For more information on how to use SRIOV together with mounting hugepages in K8 cluster please refer to:
Another quick way for launching a container is by using "host" network attachment specified in next step.
Run a container
Find the newly created Docker image name:
CLI Command
docker image list
Run Docker image:
CLI Command
docker run -itd --privileged -v /mnt/huge_c0/:/dev/hugepages/ --net=host --name=<container_name> <docker_image_name>
DPDK libraries are located inside the container under /usr/src/dpdk-<version>/. This directory contains compiled Nvidia PMD with defaults apps such as testpmd.
|
Amir Zeidner For the past several years, Amir has worked as a Solutions Architect primarily in the Telco space, leading advanced solutions to answer 5G, NFV, and SDN networking infrastructures requirements. Amir’s expertise in data plane acceleration technologies, such as Accelerated Switching and Network Processing (ASAP²) and DPDK, together with a deep knowledge of open source cloud-based infrastructures, allows him to promote and deliver unique end-to-end NVIDIA Networking solutions throughout the Telco world. |