Skip to navigation

Prerequisites: NVIDIA Driver, CUDA, Docker, and NVIDIA Container Toolkit Installation for amd64 / x86_64 Systems

View as Markdown

Introduction

If your compute host uses amd64 or x86_64 CPU architecture, follow the steps below to install the tools below based on the requirements described in the Prerequisites. Ensure you have the following:

  • NVIDIA Driver
  • CUDA
  • Docker
  • NVIDIA Container Toolkit

Installation Steps

  1. Find and download installation package for driver 580+

    a. On your local machine (with browser), visit the NVIDIA Drivers download page, and observe the fields in the ‘Manual Driver Search’ dialogue box.

    b. Enter your system information:

    • For systems with x86_64 (amd64) CPU architecture, Ubuntu 24.04, and datacenter GPUs H100, H200, A100, L40S
    Product CategoryData Center / Tesla
    Product SeriesH-Series, A-Series, or L-Series
    ProductH100, H200, A100
    OSLinux 64-bit Ubuntu 24.04
    CUDA Toolkit Version13.0
    LanguageEnglish (US)
    • For systems with x86_64 (amd64) CPU architecture, Ubuntu 24.04, and the datacenter GPU B200
    Product CategoryData Center / Tesla
    Product SeriesHGX-Series
    ProductHGX B200
    OSLinux 64-bit Ubuntu 24.04
    CUDA Toolkit Version13.0
    LanguageEnglish (US)

    c. Click Find to find driver version 580 or higher

    d. On the results page, click View

    e. On the next page, right-click the Download button and select Copy Link Address

    f. If Linux 64-bit Ubuntu 24.04 was selected, the link will look like

    https://us.download.nvidia.com/tesla/580.105.08/nvidia-driver-local-repo-ubuntu2404-580.105.08_1.0-1_amd64.deb

    but if Linux 64-bit was selected, the link will appear like

    https://us.download.nvidia.com/tesla/580.105.08/NVIDIA-Linux-x86_64-580.105.08.run

Some distributions like Ubuntu, Debian, or RHEL, have distribution specific packages, indicated with extensions .deb, .deb, and .rpm respectively. For other distributions, use the .run installer.

  1. Direct driver URLs for common configurations:

    For Ubuntu 24.04 (Noble):

    # Driver 580.105.08 for H100/H200/B200/A100/L40S on an x86_64(amd64) system
    https://us.download.nvidia.com/tesla/580.105.08/nvidia-driver-local-repo-ubuntu2404-580.105.08_1.0-1_amd64.deb

    For Ubuntu 22.04 (Jammy):

    # Driver 580.105.08 for H100/H200/B200/A100/L40S on an x86_64(amd64) system
    https://us.download.nvidia.com/tesla/580.105.08/nvidia-driver-local-repo-ubuntu2204-580.105.08_1.0-1_amd64.deb

    For RHEL 8/Rocky Linux 8:

    # Driver 580.105.08 for H100/H200/B200/A100/L40S
    https://us.download.nvidia.com/tesla/580.105.08/nvidia-driver-local-repo-rhel8-580.105.08-1.0-1.x86_64.rpm

Always check the NVIDIA Driver Downloads page for the latest driver version compatible with your GPU and OS.

The following commands use the $OS_VERSION environment variable set in step 1 to automatically use the correct OS version for your system. This ensures commands are copy-pastable regardless of your Ubuntu version.

  1. (Optional but recommended) Check current driver and purge old drivers if needed:
# Check current driver version
nvidia-smi --query-gpu=driver_version --format=csv,noheader 2>/dev/null || echo "No driver installed"
# If you have an older driver (< 580), purge it to prevent conflicts
sudo apt-get remove --purge nvidia-* -y
sudo apt-get autoremove -y

This step prevents driver library version conflicts. If you have an existing NVIDIA driver older than version 580, we recommend purging it before installing the new driver.

  1. Use wget to download the driver on your remote machine:
# Download driver using OS_VERSION environment variable
# For Ubuntu (automatically uses correct version: 2204, 2404, etc.)
wget https://us.download.nvidia.com/tesla/580.105.08/nvidia-driver-local-repo-ubuntu${OS_VERSION}-580.105.08_1.0-1_${CPU_ARCH}.deb

The ${OS_VERSION} variable holds your OS version (e.g., 2204 for Ubuntu 22.04, 2404 for Ubuntu 24.04), and the ${CPU_ARCH} holds your CPU architecture, making this command copy-pastable across Ubuntu versions.

  1. Install the local repository using dpkg (for Ubuntu/Debian):
sudo dpkg -i nvidia-driver-local-repo-ubuntu${OS_VERSION}-580.105.08_1.0-1_${CPU_ARCH}.deb

For RHEL/CentOS/Rocky Linux:

sudo rpm -i nvidia-driver-local-repo-rhel8-580.105.08-1.0-1.${CPU_ARCH}.rpm
  1. Update package lists and install the driver:

For Ubuntu/Debian:

# Copy the GPG key (automatically uses correct OS version)
sudo cp /var/nvidia-driver-local-repo-ubuntu${OS_VERSION}-580.105.08/nvidia-driver-local-*-keyring.gpg /usr/share/keyrings/
# Update package cache
sudo apt-get update
# Install the driver
sudo apt-get install -y cuda-drivers

For RHEL/CentOS/Rocky Linux:

# Update package cache
sudo dnf clean all
sudo dnf makecache
# Install the driver
sudo dnf install -y cuda-drivers
  1. After installation, reboot to load the new driver:
sudo reboot
  1. After reboot, verify the driver is installed correctly:
nvidia-smi

You should see output showing your GPU(s) and driver version 580 or higher with CUDA version 13.0 or higher.

Example expected output:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 580.105.08    Driver Version: 580.105.08    CUDA Version: 13.0   |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA H100 PCIe    Off  | 00001E:00:00.0  Off |                    0  |
| N/A   30C    P0    68W / 350W |      0MiB / 81559MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+

Docker Installation

Verify that docker is installed with version >=23.0.1 with the command below.

docker --version
# Example output:
# Docker version 29.1.3, build f52814d

If docker is not installed, or is installed but the version does not meet requirements, use the instructions linked below to do the installation.

NVIDIA Container Toolkit Installation

Verify that the NVIDIA Container Toolkit is installed, and meets the minimum version requirement, with the command below.

nvidia-container-cli --version

If the NVIDIA Container Toolkit is not installed, or does not meet the minimum version requirement, you must install it.

To install and configure the NVIDIA Container Toolkit

  1. Follow the instructions at Installing the NVIDIA Container Toolkit.

  2. Configure the NVIDIA Container Toolkit using the instructions at configuring-docker.

Verifying the NVIDIA Driver, Docker, and NVIDIA Container Toolkit Stack

Verify your NVIDIA Driver, docker, and NVIDIA Container Toolkit configuration by running the command below sourced from running-a-sample-workload-with-docker.

sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

Example output:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 580.82.07    Driver Version: 580.82.07    CUDA Version: 13.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| 41%   30C    P8     1W / 260W |   2244MiB / 11264MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

For more information on enumerating multi-GPU systems, refer to the NVIDIA Container Toolkit’s GPU Enumeration Docs