QSG of Building Docker Image with Compiled Nvidia Network DPDK PMD

Created on Jul 11, 2019

image2019-7-11_16-50-3.png

This post presents how to build a Docker container with compiled Nvidia Network DPDK PMD.

References

Hypervisor Prerequisite

  • Install Docker version >= 1.9.

  • Install Nvidia networks rdma-core package:

CLI Command

Copy
Copied!
            

yum install -y rdma-core-devel

Creating Docker Container Image

Download Dockerfile from Github and build the image locally on the hypervisor.

CLI Commands

Copy
Copied!
            

git clone https://github.com/Mellanox/mlnx_docker_dpdk cd mlnx_docker_dpdk/ ## Example for OS: CentOS-7.7.1908 DPDK Version: 20.02 MOFED Version: 5.1-2.3.7.1 ## docker build --build-arg OS_VER=7.7.1908 --build-arg DPDK_VER=20.02 --build-arg OFED_VER=5.1-2.3.7.1 -t centos/<image_name>:<version_name> .

  • OS_VER - CentOS image version

  • DPDK_VER - DPDK version

  • OFED_VER - Nvidia OFED version. Not specifying MOFED version will install default MOFED upstream packages.

  • <image_name> and <version_name> - Can be any user defined name

  • <image_name> and <version_name> can be any user-defined name

Warning

Note: Default master branch supports only CentOS base OS.

ubuntu18.04, ubuntu14.04 , ubuntu16.04 , rhel7.2 , rhel7.3 can be used with different Dockerfile which is available in this git project under unique bran

Running Docker Container with Nvidia Network DPDK PMD

Define and mount hugepages on the hypervisor

Allocate 1G size of hugepages:

CLI Command

Copy
Copied!
            

echo 4 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages

OR

Allocate 2Mb size of hugepages:

CLI Commands

Copy
Copied!
            

echo 2048 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages   Mount hugespages:   mount -t hugetlbfs -o pagesize=1G none /dev/hugepages   mkdir -p /mnt/huge_c0   mount -t hugetlbfs -o pagesize=1G none  /mnt/huge_c0/

Warning

This example exhibits allocation of 4Gb total hugepages for both 2Mb and 1Gb hugepages sizes. You may define different amounts relative to the free amount of memory on your hypervisor.

Define the network for the container

The recommended type of network attachment is SRIOV.

For more information on how to use SRIOV together with mounting hugepages in K8 cluster please refer to:

Another quick way for launching a container is by using "host" network attachment specified in next step.

Run a container

Find the newly created Docker image name:

CLI Command

Copy
Copied!
            

docker image list

Run Docker image:

CLI Command

Copy
Copied!
            

docker run -itd --privileged -v /mnt/huge_c0/:/dev/hugepages/ --net=host --name=<container_name> <docker_image_name>

Warning

DPDK libraries are located inside the container under /usr/src/dpdk-<version>/. This directory contains compiled Nvidia PMD with defaults apps such as testpmd.

me.JPG

Amir Zeidner

For the past several years, Amir has worked as a Solutions Architect primarily in the Telco space, leading advanced solutions to answer 5G, NFV, and SDN networking infrastructures requirements. Amir’s expertise in data plane acceleration technologies, such as Accelerated Switching and Network Processing (ASAP²) and DPDK, together with a deep knowledge of open source cloud-based infrastructures, allows him to promote and deliver unique end-to-end NVIDIA Networking solutions throughout the Telco world.

Last updated on Sep 12, 2023.