Installing the NVIDIA Container Toolkit

Installation
Configuration
Next Steps

Installation

Prerequisites

Install the NVIDIA GPU driver for your Linux distribution. NVIDIA recommends installing the driver by using the package manager for your distribution.

For information about installing the driver with a package manager, refer to the NVIDIA Driver Installation Quickstart Guide.

Alternatively, you can install the driver by downloading a .run installer. Refer to the NVIDIA Official Drivers page.

Installing with Apt

Configure the production repository:

$ curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

Optionally, configure the repository to use experimental packages:

$ sed -i -e '/experimental/ s/^#//g' /etc/apt/sources.list.d/nvidia-container-toolkit.list

Update the packages list from the repository:
```
$ sudo apt-get update
```

Install the NVIDIA Container Toolkit packages:

$ sudo apt-get install -y nvidia-container-toolkit

Installing with Yum or Dnf

Configure the production repository:

$ curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \
  sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo

Optionally, configure the repository to use experimental packages:

$ sudo yum-config-manager --enable nvidia-container-toolkit-experimental

Install the NVIDIA Container Toolkit packages:

$ sudo yum install -y nvidia-container-toolkit

Installing with Zypper

Configure the production repository:

$ sudo zypper ar https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo

Optionally, configure the repository to use experimental packages:

$ sudo zypper modifyrepo --enable nvidia-container-toolkit-experimental

Install the NVIDIA Container Toolkit packages:

$ sudo zypper --gpg-auto-import-keys install -y nvidia-container-toolkit

Configuration

Prerequisites

You installed a supported container engine (Docker, Containerd, CRI-O, Podman).
You installed the NVIDIA Container Toolkit.

Configuring Docker

Configure the container runtime by using the nvidia-ctk command:
```
$ sudo nvidia-ctk runtime configure --runtime=docker
```
The nvidia-ctk command modifies the /etc/docker/daemon.json file on the host. The file is updated so that Docker can use the NVIDIA Container Runtime.
Restart the Docker daemon:
```
$ sudo systemctl restart docker
```

Rootless mode

To configure the container runtime for Docker running in Rootless mode, follow these steps:

Configure the container runtime by using the nvidia-ctk command:

$ nvidia-ctk runtime configure --runtime=docker --config=$HOME/.config/docker/daemon.json

Restart the Rootless Docker daemon:
```
$ systemctl --user restart docker
```
Configure /etc/nvidia-container-runtime/config.toml by using the sudo nvidia-ctk command:
```
$ sudo nvidia-ctk config --set nvidia-container-cli.no-cgroups --in-place
```

Configuring containerd (for Kubernetes)

Configure the container runtime by using the nvidia-ctk command:
```
$ sudo nvidia-ctk runtime configure --runtime=containerd
```
The nvidia-ctk command modifies the /etc/containerd/config.toml file on the host. The file is updated so that containerd can use the NVIDIA Container Runtime.
Restart containerd:
```
$ sudo systemctl restart containerd
```

Configuring containerd (for nerdctl)

No additional configuration is needed. You can just run nerdctl run --gpus=all, with root or without root. You do not need to run the nvidia-ctk command mentioned above for Kubernetes.

Configuring CRI-O

Configure the container runtime by using the nvidia-ctk command:
```
$ sudo nvidia-ctk runtime configure --runtime=crio
```
The nvidia-ctk command modifies the /etc/crio/crio.conf file on the host. The file is updated so that CRI-O can use the NVIDIA Container Runtime.
Restart the CRI-O daemon:
```
$ sudo systemctl restart crio
```

Configuring Podman

For Podman, NVIDIA recommends using CDI for accessing NVIDIA devices in containers.

Next Steps

Running a Sample Workload