Installing BaseOS 9 Software#

The NVIDIA BaseOS 9 software stack provides a turnkey solution for running AI and analytics workloads. It comprises platform-specific configurations, diagnostic and monitoring tools, and drivers that are required for a stable, tested, and supported OS to run AI, machine learning, and analytics applications on DGX platforms.

Ubuntu 26.04#

The following sections describe the prerequisites, installation, and configuration required to deploy the BaseOS software stack on your system running Ubuntu 26.04.

Prerequisites#

Ubuntu Software Requirements#

The BaseOS software stack requires the following:

  • Ubuntu 26.04

  • NVIDIA Linux kernel 7.0.0-1010-nvidia for supported NVIDIA GPUs (x86_64)

Access to Software Repositories#

The BaseOS software stack is distributed through internet-accessible repositories.

  • Air-gapped installation

    If your system cannot connect to the internet, refer to Air-Gapped Installations for instructions on installing and upgrading software on air-gapped systems.

  • Proxy installation

    If you use a proxy server, refer to Network Configurations for setup instructions.

Installing the BaseOS Software Stack#

This section requires that you have already installed Ubuntu on your system and rebooted the system.

Attention

By installing the BaseOS Software Stack, you are confirming that you have read and agree to be bound by the NVIDIA Software License Agreement. You are also confirming that you understand that any pre-release software and materials available that you elect to install in a DGX platform may not be fully functional, may contain errors or design flaws, and may have reduced or different security, privacy, availability, and reliability standards relative to commercial versions of NVIDIA software and materials, and that you use pre-release versions at your risk.

Enabling NVIDIA Repositories#

The following steps enable the NVIDIA repositories.

  1. Enable the NVIDIA repositories by extracting the repository information.

    These steps add the URIs and configuration preferences to control the package versions that are installed to the /etc/apt directory and the GNU Privacy Guard (GPG) keys for the NVIDIA repositories in the /usr/share/keyrings directory. Download the archive with strict curl flags before extracting it; do not process downloaded content directly as root.

    • For x86_64 systems:

      curl --proto '=https' --tlsv1.2 -fsSLo /tmp/dgx-repo-files.tgz \
  https://repo.download.nvidia.com/baseos/9/ubuntu/resolute/x86_64/dgx-repo-files.tgz

    • For ARM64 systems:

      curl --proto '=https' --tlsv1.2 -fsSLo /tmp/dgx-repo-files.tgz \
  https://repo.download.nvidia.com/baseos/9/ubuntu/resolute/arm64/dgx-repo-files.tgz

    Check the file contents:

    sudo tar tzf /tmp/dgx-repo-files.tgz

    The output shows the following contents:

    ./usr/
./usr/share/
./usr/share/keyrings/
./usr/share/keyrings/nvidia-hpcsdk-archive-keyring.gpg
./usr/share/keyrings/cuda_debian_prod.gpg
./usr/share/keyrings/docker-key.gpg
./usr/share/keyrings/nvidia-doca-debian-gpg-public-key.gpg
./usr/share/keyrings/ai-workbench-desktop-key.gpg
./usr/share/keyrings/GPG-KEY-Mellanox.gpg
./usr/share/keyrings/dgx_debian_prod.gpg
./etc/
./etc/apt/
./etc/apt/sources.list.d/
./etc/apt/sources.list.d/dgx.sources
./etc/apt/sources.list.d/nvidia.sources
./etc/apt/preferences.d/
./etc/apt/preferences.d/nvidia-dgx

    Verify that the archive contents match the expected repository files. If they do not, do not continue; confirm that the download command used matches the download command in the previous step and that the archive was not corrupted.

    After verifying the archive contents, extract the archive and remove the temporary file:

    sudo tar xzf /tmp/dgx-repo-files.tgz -C /
rm -f /tmp/dgx-repo-files.tgz

  2. Update the internal APT database with the latest version information of all packages.

    sudo apt update

  3. Recommended: Upgrade all software packages to the latest versions.

    sudo apt upgrade

    sudo reboot

Installing the GPU Driver#

DGX OS installs the supported NVIDIA open GPU driver during the OS installation. For the current supported driver and software-stack versions, refer to Current Software Versions in the DGX OS Release Notes.

The ISO includes the nvidia-open metapackage and its corresponding nvidia-driver-open dependency in its local package pool. The platform installation package list requests nvidia-open, and the installer installs it from the ISO with the other platform software.

The nvidia-open metapackage depends on nvidia-driver-open, which in turn installs the matching NVIDIA user-space libraries and the nvidia-dkms-open package. The ISO includes the nvidia-dkms-open package, which contains the open GPU driver source code and DKMS build instructions. When the package is installed, DKMS builds the GPU kernel module for the NVIDIA kernel installed on the system.

Administrators do not need to select a driver branch during installation. To remain on the supported configuration, update the DGX OS/BaseOS software stack as a unit rather than independently changing the NVIDIA GPU driver.

  1. The DGX OS ISO installer installs the NVIDIA open GPU kernel modules and associated software automatically; therefore, it is not necessary to install these packages separately after an ISO installation.

    Driver and associated software installed by the ISO#

    Package

    Installation method

    nvidia-open, nvidia-driver-open, and nvidia-dkms-open

    The platform package list requests nvidia-open from the ISO package pool. Its dependencies install the driver metapackage and the DKMS package, which builds the GPU kernel module for the installed NVIDIA kernel.

    libnvidia-nscq, nvidia-fabricmanager, and nvidia-persistenced

    Installed from the platform-specific package list for supported DGX servers.

    nvidia-modprobe and datacenter-gpu-manager-4-cuda13

    Included in the BaseOS server squashfs selected by the installer.

    nvlsm and libnvsdm

    Installed from the platform-specific package list on DGX B200 and DGX B300 systems.

  2. The DGX OS ISO installer enables the NVIDIA Fabric Manager, DCGM, and NVIDIA Persistence Daemon services for supported DGX server platforms. The nvidia-fabricmanager and nvidia-persistenced package installation scripts also enable and start their services by default.

    It is safe for the automated installer to start services that have already been started. Starting these services by the installer ensures that all required services, including DCGM, are enabled in the final target system even if a package installation script was skipped, deferred, or affected by installer policy. Package scripts enable and start their own services when the packages are configured; the installer ensures the required DGX services are enabled together.

    The installer does not stop if a service-enable command fails. After the installation is complete, verify that the services are enabled and running:

    systemctl is-enabled nvidia-fabricmanager nvidia-persistenced nvidia-dcgm
systemctl status nvidia-fabricmanager nvidia-persistenced nvidia-dcgm

  3. Reboot the system to load the drivers and to update system configurations.

    1. Reboot the system.

      sudo reboot

    2. After the system has rebooted, verify that the drivers have been loaded and are managing the NVIDIA devices.

      nvidia-smi

      The output should show all available GPUs and Persistence-Mode On:

      +--------------------------------------------------------------------------------+
| NVIDIA-SMI 595.91.07   Driver Version: 595.91.07    CUDA Version: 13.3.1       |
|--------------------------------+-----------------------+-----------------------+
| GPU Name         Persistence-M | Bus-Id         Disp.A | Volatile Uncorr.  ECC |
| Fan Temp Perf    Pwr:Usage/Cap |          Memory-Usage | GPU-Util   Compute M. |
|                                |                       |                MIG M. |
|================================+=======================+=======================|
|   0  NVIDIA B200           On  | 000000000:1B:00.0 Off |                     0 |
| N/A   33C    P0   188W / 1000W |      0MiB / 183359MiB |   0%          Default |
|                                |                       |              Disabled |
+--------------------------------+-----------------------+-----------------------+
|   1  NVIDIA B200           On  | 000000000:43:00.0 Off |                     0 |
| N/A   34C    P0   190W / 1000W |      0MiB / 183359MiB |   0%          Default |
|                                |                       |              Disabled |
+--------------------------------+-----------------------+-----------------------+
...
+--------------------------------+-----------------------+-----------------------+
|   7  NVIDIA B200           On  | 000000000:DF:00.0 Off |                     0 |
| N/A   38C     P0  197W / 1000W |      0MiB / 183359MiB |   0%          Default |
|                                |                       |              Disabled |
+--------------------------------+-----------------------+-----------------------+
+--------------------------------------------------------------------------------+
| Processes:                                                                     |
| GPU   GI   CI          PID     Type      Process name               GPU Memory |
|       ID   ID                                                       Usage      |
|================================================================================|
| No running processes found                                                     |
+--------------------------------------------------------------------------------+

Installing NVIDIA System Configurations and Tools#

The NVIDIA BaseOS Software Stack includes system-specific configurations and tools to leverage the advanced DGX features. They are provided from NVIDIA repositories in the form of software packages that can be installed on top of a typical Ubuntu installation. All system-specific software components are bundled into metapackages specific to a system:

  • nvidia-system-core

  • nvidia-system-extra

  • nvidia-system-utils

The following steps enable the NVIDIA repositories and install the system-specific packages.

  1. Enable the NVIDIA repositories by extracting the repository information.

    This step adds the URIs and configuration preferences to control the package versions that are installed to the /etc/apt directory and the GNU Privacy Guard (GPG) keys for the NVIDIA repositories in the /usr/share/keyrings directory. Download the archive with strict curl flags before extracting it; do not process downloaded content directly as root.

    • For x86_64 systems:

      curl --proto '=https' --tlsv1.2 -fsSLo /tmp/dgx-repo-files.tgz \
  https://repo.download.nvidia.com/baseos/9/ubuntu/resolute/x86_64/dgx-repo-files.tgz

    • For ARM64 systems:

      curl --proto '=https' --tlsv1.2 -fsSLo /tmp/dgx-repo-files.tgz \
  https://repo.download.nvidia.com/baseos/9/ubuntu/resolute/arm64/dgx-repo-files.tgz

    Check the file contents:

    sudo tar tzf /tmp/dgx-repo-files.tgz

    The output shows the following contents:

    ./usr/
./usr/share/
./usr/share/keyrings/
./usr/share/keyrings/nvidia-hpcsdk-archive-keyring.gpg
./usr/share/keyrings/cuda_debian_prod.gpg
./usr/share/keyrings/docker-key.gpg
./usr/share/keyrings/nvidia-doca-debian-gpg-public-key.gpg
./usr/share/keyrings/ai-workbench-desktop-key.gpg
./usr/share/keyrings/GPG-KEY-Mellanox.gpg
./usr/share/keyrings/dgx_debian_prod.gpg
./etc/
./etc/apt/
./etc/apt/sources.list.d/
./etc/apt/sources.list.d/dgx.sources
./etc/apt/sources.list.d/nvidia.sources
./etc/apt/preferences.d/
./etc/apt/preferences.d/nvidia-dgx

    Verify that the archive contents match the expected repository files. If they do not, do not continue; confirm that the download command used matches the download command in the previous step and that the archive was not corrupted.

    After verifying the archive contents, extract the archive and remove the temporary file:

    sudo tar xzf /tmp/dgx-repo-files.tgz -C /
rm -f /tmp/dgx-repo-files.tgz

  2. Update the internal APT database with the latest version information of all packages.

    sudo apt update

  3. Recommended: Upgrade all software packages to the latest versions.

    sudo apt upgrade

    sudo reboot

  4. Install the following NVIDIA system tools and utilities.

    sudo apt install nvidia-system-core
sudo apt install nvidia-system-utils
sudo apt install nvidia-system-extra

  5. Install the DGX system tools and utilities.

    • For DGX servers, install the dgx-system-utils package.

      sudo apt install dgx-system-utils

    • For DGX Spark, install the nvidia-system-station package for a complete desktop, including the GNOME desktop, Xorg, and other desktop packages.

      sudo apt install nvidia-system-station

  6. Install the linux-tools package for your kernel.

    • For x86_64 systems:

      sudo apt install -y linux-tools-nvidia

    • For ARM64 systems like DGX GB200 and DGX GB300:

      sudo apt install -y linux-tools-nvidia-64k-hwe-24.04

    • For ARM64 systems like DGX Spark:

      sudo apt install -y linux-tools-nvidia-hwe-24.04

  7. Recommended: Disable unattended upgrades.

    Ubuntu periodically checks for security and other bug fixes and automatically updates software packages typically overnight. Because this can be disruptive, you should regularly check for updates and install them manually.

    sudo apt purge -y unattended-upgrades

    Note

    DGX OS uses DMA-BUF for supported GPUDirect RDMA configurations and does not require the nvidia_peermem kernel module. The nvidia-peermem module is retained only for legacy PeerDirect consumers and must not be loaded by default.

    Earlier releases installed nvidia-peermem-loader to arrange for nvidia_peermem to load automatically. The nvidia-peermem-loader package is not needed in the current release: it is an empty transitional package that can be safely removed. Accordingly, the nvidia-base TuneD profile does not load nvidia_peermem.

Configuring Data Drives#

The data drives in the DGX systems can be configured as RAID 0 or RAID 5. RAID 0 provides the maximum storage capacity and performance but does not provide any redundancy.

RAID 0 is often used for data caching. You can use cachefilesd to provide a cache for NFS shares.

Important

You can change the RAID level later, but this destroys the data on those drives.

You can configure the RAID array during the Ubuntu installation. If you have already configured the RAID array during the Ubuntu installation, you can skip the first step and go to step 2.

  1. Configure the /raid partition.

    All DGX systems support RAID 0 and RAID 5 arrays.

    • To create a RAID 0 array:

      sudo /usr/bin/configure_raid_array.py -c -f

    • To create a RAID 5 array:

      sudo /usr/bin/configure_raid_array.py -c -f -5

    The command creates the /raid mount point and RAID array and adds a corresponding entry in /etc/fstab.

  2. (Optional) Install tools for managing the self-encrypting drives (SED) for the data drives on the DGX A100, DGX H100/H200, DGX B200, DGX B300, DGX GB200, or DGX GB300.

    Refer to Managing Self-Encrypting Drives for more information.

  3. (Optional) To use your RAID array for read caching of NFS mounts, you can install cachefilesd and set the cachefs option for an NFS share.

    1. Install cachefilesd and nvidia-conf-cachefilesd.

      This step updates the cachefilesd configuration to use the /raid partition.

      sudo apt install -y cachefilesd nvidia-conf-cachefilesd

      Both cachefilesd and nvidia-conf-cachefilesd might already be installed with the nvidia-system-utils metapackage.

    2. Enable caching on all NFS shares you want to cache by setting the fsc flag.

      Edit /etc/fstab and add the fsc flag to the mount options as shown in this example.

      <nfs_server>:<export_path> /mnt nfs rw,noatime,rsize=32768,wsize=32768,nolock,tcp,intr,fsc,nofail 0 0

    3. Mount the NFS share.

      If the share is already mounted, use the remount option.

      mount -o remount <mount-point>

    4. To validate that caching is enabled, issue the following command.

      cat /proc/fs/nfsfs/volumes

      Look for the text FSC=yes in the output of the command. NFS is mounted with caching enabled upon subsequent reboot cycles.

Installing the DOCA-OFED Package#

DGX systems include high-performance network cards for InfiniBand or Ethernet connectivity. NVIDIA DOCA™ OFED is equivalent to MLNX_OFED, providing the same kernel drivers, user space libraries, and management tools for NVIDIA networking products. For more information about DOCA-OFED, refer to the What Is DOCA-OFED section in MLNX_OFED to DOCA-OFED Transition Guide. For complete installation information, refer to the NVIDIA DOCA Installation Guide for Linux.

Install the latest version of NVIDIA DOCA-OFED software by adding the NVIDIA DOCA-OFED repository to your system.

  1. Install the nvidia-repositories and nvidia-repo-keys packages to automate secure package access.

    sudo apt update
sudo apt install nvidia-repositories nvidia-repo-keys -y

  2. Refresh your package list to include the newly added DOCA repository.

    sudo apt update

  3. Perform a full upgrade to ensure all dependencies match the new repository.

    sudo apt full-upgrade -y

  4. Install the MLNX drivers.

    sudo apt install nvidia-system-mlnx-drivers -y

Installing Docker and the NVIDIA Container Toolkit#

Containers provide isolated environments with a full filesystem of the required software for specific applications. To use the NVIDIA-provided containers for AI and other frameworks on the DGX system and GPUs, you need to install Docker and the NVIDIA Container Toolkit. It takes care of providing access to the GPUs to the software running inside the container.

These tools are also required by the Firmware Update Containers for upgrading the system firmware.

  1. Install Docker CE, NVIDIA Container Toolkit, and optimizations for a typical DL workload.

    The docker-ce, nvidia-container-toolkit, and nv-docker-options packages might already be installed with the nvidia-system-utils and nvidia-system-extra metapackages.

    sudo apt install -y docker-ce nvidia-container-toolkit nv-docker-options

  2. Restart the Docker daemon.

    sudo systemctl restart docker

To validate the installation, run a container and check that it can access the GPUs. The following instructions assume that the NVIDIA GPU driver has been installed and loaded.

Note

This validation downloads a container from the NGC registry. The system must have internet access.

  1. Run the following command to start a container and run the nvidia-smi tool inside the container:

    sudo docker run --gpus=all --rm nvcr.io/nvidia/cuda:12.6.2-base-ubuntu24.04 nvidia-smi

  2. Verify that the output shows all available GPUs and has Persistence-Mode set to On.

Additional Software Installed by BaseOS#

The Ubuntu and NVIDIA repositories provide many additional software packages for a variety of applications. The BaseOS Installer, for example, installs several additional software packages to aid system administrators and developers that are not installed by default.

Follow these steps to install the additional software packages included with the BaseOS Installer.

Some of these packages might already be installed with the nvidia-system-extra metapackage.

  1. Install additional software for system administration tasks:

    sudo apt install -y chrpath cifs-utils fping gdisk iperf ipmitool lsscsi net-tools nfs-common quota rasdaemon pm-utils samba-common samba-libs sysstat vlan

  2. Install additional software for development tasks:

    sudo apt install -y build-essential automake bison cmake flex gcc-multilib gdb g++-multilib libelf-dev libltdl-dev m4 swig

The NVIDIA CUDA Developer repository provides an easy mechanism to deploy NVIDIA tools and libraries, such as the CUDA Toolkit, cuDNN, or NCCL.