Created on Jul 11, 2019
This post presents how to build a Docker container with compiled Nvidia Network DPDK PMD.
- Nvidia Network MOFED
- Docker installation reference - Install Docker | Docker Documentation
- Install Docker version >= 1.9.
- Install Nvidia networks rdma-core package:
yum install -y rdma-core-devel
Creating Docker Container Image
Download Dockerfile from Github and build the image locally on the hypervisor.
git clone https://github.com/Mellanox/mlnx_docker_dpdk cd mlnx_docker_dpdk/ ## Example for OS: CentOS-7.7.1908 DPDK Version: 20.02 MOFED Version: 5.1-22.214.171.124 ## docker build --build-arg OS_VER=7.7.1908 --build-arg DPDK_VER=20.02 --build-arg OFED_VER=5.1-126.96.36.199 -t centos/<image_name>:<version_name> .
OS_VER - CentOS image version
DPDK_VER - DPDK version
OFED_VER - Nvidia OFED version. Not specifying MOFED version will install default MOFED upstream packages.
<image_name> and <version_name> - Can be any user defined name
- <image_name> and <version_name> can be any user-defined name
Note: Default master branch supports only CentOS base OS.
ubuntu18.04, ubuntu14.04 , ubuntu16.04 , rhel7.2 , rhel7.3 can be used with different Dockerfile which is available in this git project under unique bran
Running Docker Container with Nvidia Network DPDK PMD
Define and mount hugepages on the hypervisor
Allocate 1G size of hugepages:
echo 4 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages
Allocate 2Mb size of hugepages:
echo 2048 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages Mount hugespages: mount -t hugetlbfs -o pagesize=1G none /dev/hugepages mkdir -p /mnt/huge_c0 mount -t hugetlbfs -o pagesize=1G none /mnt/huge_c0/
This example exhibits allocation of 4Gb total hugepages for both 2Mb and 1Gb hugepages sizes. You may define different amounts relative to the free amount of memory on your hypervisor.
Define the network for the container
The recommended type of network attachment is SRIOV.
For more information on how to use SRIOV together with mounting hugepages in K8 cluster please refer to:
Another quick way for launching a container is by using "host" network attachment specified in next step.
Run a container
Find the newly created Docker image name:
docker image list
Run Docker image:
docker run -itd --privileged -v /mnt/huge_c0/:/dev/hugepages/ --net=host --name=<container_name> <docker_image_name>
DPDK libraries are located inside the container under /usr/src/dpdk-<version>/. This directory contains compiled Nvidia PMD with defaults apps such as testpmd.
For the past several years, Amir has worked as a Solutions Architect primarily in the Telco space, leading advanced solutions to answer 5G, NFV, and SDN networking infrastructures requirements. Amir’s expertise in data plane acceleration technologies, such as Accelerated Switching and Network Processing (ASAP²) and DPDK, together with a deep knowledge of open source cloud-based infrastructures, allows him to promote and deliver unique end-to-end NVIDIA Networking solutions throughout the Telco world.