Installation

Prerequisites

System requirements before installing AODT components
View as Markdown

Complete these steps before installing the client or worker. The requirements depend on your deployment type.

Deployment types

DeploymentWhat it meansPrerequisites on this page
ColocatedClient and worker run on the same GPU-enabled Linux machine.The prerequisites on this page apply to that machine.
Non-colocatedClient and worker run on different machines.The prerequisites on this page apply to the worker host only. The client machine does not require a GPU or the software listed here.

For non-colocated setups, install the items below on the worker system, then install the client on a separate machine using the OS-specific steps in Client Installation.

Summary

The worker host must provide:

  • Ubuntu Linux 22.04
  • An NVIDIA GPU (RTX 6000 Ada, L40, A100, or H100)
  • NVIDIA driver 575.57.0 (576.80 on AWS)
  • Docker with Docker Compose v2 (CLI plugin) 2.3 or newer
  • NVIDIA Container Toolkit

The legacy standalone docker-compose v1 is not supported. Use the Docker Compose v2 CLI plugin (docker compose).

Install Docker

Install Docker from the official repository. This includes the docker-compose-plugin package.

$sudo apt-get install -y ca-certificates curl
$sudo install -m 0755 -d /etc/apt/keyrings
$sudo mkdir -p /etc/apt/sources.list.d
$sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
$sudo chmod a+r /etc/apt/keyrings/docker.asc
$echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | sudo tee /etc/apt/sources.list.d/docker.list
$sudo apt-get update
$sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
$sudo usermod -aG docker $USER

Log out and back in after usermod so Docker commands work without sudo.

Verify the installation:

$docker --version
$docker compose version

Install NVIDIA Container Toolkit

The worker requires GPU access inside Docker containers. Install and configure the NVIDIA Container Toolkit:

$curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
$curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
>  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
>  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
$sudo apt-get update
$sudo apt-get install -y nvidia-container-toolkit
$sudo nvidia-ctk runtime configure --runtime=docker
$sudo systemctl restart docker

Verify that the host can see the GPU:

$nvidia-smi

Install NVIDIA driver

Install the NVIDIA driver if it is not already present. Skip this step if nvidia-smi reports a supported driver version.

$# List available driver packages
$apt-cache search nvidia-driver
$
$# Install using the driver series number (e.g. nvidia-driver-535).
$# Do NOT supply a patch-level version number like 590.48.01.
$sudo apt-get install -y nvidia-driver-<driver-series>
$sudo reboot

After reboot, confirm the driver is active:

$nvidia-smi

See the CUDA compatibility guide for driver and toolkit version requirements.

Optional: CUDA toolkit

CUDA Toolkit is not required for the worker and it is optional for running the client. It is required on the client only when you want colocated LOCAL_IPC transport for direct GPU memory access. See Client Installation for CUDA setup on the client.

Next steps

When all prerequisites are installed and verified:

  1. Download the AODT package from NGC.
  2. Install the client and worker.
  3. Follow the Quickstart guide to validate your deployment.