Installing the NVIDIA Container Toolkit
Installation
Prerequisites
Install the NVIDIA GPU driver for your Linux distribution. NVIDIA recommends installing the driver by using the package manager for your distribution.
For information about installing the driver with a package manager, refer to the NVIDIA Driver Installation Quickstart Guide.
Alternatively, you can install the driver by downloading a .run
installer.
Refer to the NVIDIA Official Drivers page.
Installing with Apt
Configure the production repository:
$ curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \ && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \ sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \ sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
Optionally, configure the repository to use experimental packages:
$ sed -i -e '/experimental/ s/^#//g' /etc/apt/sources.list.d/nvidia-container-toolkit.list
Update the packages list from the repository:
$ sudo apt-get update
Install the NVIDIA Container Toolkit packages:
$ sudo apt-get install -y nvidia-container-toolkit
Installing with Yum or Dnf
Configure the production repository:
$ curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \ sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
Optionally, configure the repository to use experimental packages:
$ sudo yum-config-manager --enable nvidia-container-toolkit-experimental
Install the NVIDIA Container Toolkit packages:
$ sudo yum install -y nvidia-container-toolkit
Installing with Zypper
Configure the production repository:
$ sudo zypper ar https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo
Optionally, configure the repository to use experimental packages:
$ sudo zypper modifyrepo --enable nvidia-container-toolkit-experimental
Install the NVIDIA Container Toolkit packages:
$ sudo zypper --gpg-auto-import-keys install -y nvidia-container-toolkit
Configuration
Prerequisites
You installed a supported container engine (Docker, Containerd, CRI-O, Podman).
You installed the NVIDIA Container Toolkit.
Configuring Docker
Configure the container runtime by using the
nvidia-ctk
command:$ sudo nvidia-ctk runtime configure --runtime=docker
The
nvidia-ctk
command modifies the/etc/docker/daemon.json
file on the host. The file is updated so that Docker can use the NVIDIA Container Runtime.Restart the Docker daemon:
$ sudo systemctl restart docker
Rootless mode
To configure the container runtime for Docker running in Rootless mode, follow these steps:
Configure the container runtime by using the
nvidia-ctk
command:$ nvidia-ctk runtime configure --runtime=docker --config=$HOME/.config/docker/daemon.json
Restart the Rootless Docker daemon:
$ systemctl --user restart docker
Configure
/etc/nvidia-container-runtime/config.toml
by using thesudo nvidia-ctk
command:$ sudo nvidia-ctk config --set nvidia-container-cli.no-cgroups --in-place
Configuring containerd (for Kubernetes)
Configure the container runtime by using the
nvidia-ctk
command:$ sudo nvidia-ctk runtime configure --runtime=containerd
The
nvidia-ctk
command modifies the/etc/containerd/config.toml
file on the host. The file is updated so that containerd can use the NVIDIA Container Runtime.Restart containerd:
$ sudo systemctl restart containerd
Configuring containerd (for nerdctl)
No additional configuration is needed.
You can just run nerdctl run --gpus=all
, with root or without root.
You do not need to run the nvidia-ctk
command mentioned above for Kubernetes.
See also the nerdctl documentation.
Configuring CRI-O
Configure the container runtime by using the
nvidia-ctk
command:$ sudo nvidia-ctk runtime configure --runtime=crio
The
nvidia-ctk
command modifies the/etc/crio/crio.conf
file on the host. The file is updated so that CRI-O can use the NVIDIA Container Runtime.Restart the CRI-O daemon:
$ sudo systemctl restart crio
Configuring Podman
For Podman, NVIDIA recommends using CDI for accessing NVIDIA devices in containers.