Installing Additional Software#

DGX OS 8 is an optimized version of the Ubuntu 26.04 Linux distribution that provides access to an extensive collection of additional software available from the Ubuntu and NVIDIA repositories. For more information about additional software available from Ubuntu, refer to Install additional applications.

Before you install additional software or upgrade installed software, refer to the Release Notes for the latest release information. To install the additional software, use the apt command.

Installing GPUDirect Storage Support#

NVIDIA Magnum IO GPUDirect Storage (GDS) enables a direct data path for direct memory access (DMA) transfers between GPU memory and storage. This software avoids using a bounce buffer through the CPU.

Note

This section applies to bare-metal DGX servers that use GPUDirect Storage. GDS can use the traditional nvidia-fs data path or the Linux PCI P2PDMA data path when the kernel, storage driver, and system topology support it. When both paths are supported for NVMe, P2PDMA takes precedence.

Use the Linux PCI P2PDMA troubleshooting and verification instructions to determine whether P2PDMA is available. P2PDMA is not supported with RAID.

Installing GDS Components#

On DGX servers (DGX B300/B200, H100/H200/H800, and A100/A800):

Install the nvidia-gds package.

Install this package before configuring the GDS data path.
```
sudo apt install nvidia-gds
```
Optional: Configure GDS to use the traditional nvidia-fs data path for NVMe.

To disable the P2PDMA preference, edit /etc/cufile.json and set both use_pci_p2pdma entries to false. Retain the other settings in the existing file.
```
{
  "properties": {
    "use_pci_p2pdma": false
  },
  "block": {
    "nvme": {
      "use_pci_p2pdma": false
    }
  }
}
```
Do not set CUFILE_USE_PCIP2PDMA=true for the GDS application; if it is already set, remove or unset it. This environment variable requests P2PDMA instead of the traditional nvidia-fs data path.

Refer to Verifying a Successful GDS Installation in the NVIDIA GPUDirect Storage Installation and Troubleshooting Guide. After changing the data path configuration, run gdscheck.py -p to verify the selected NVMe path.

Enabling Relaxed Ordering for NVMe Drives#

The Samsung PM1733 NVMe data drives in DGX A100/A800 systems support PCIe relaxed ordering for I/O operations. Relaxed ordering enables the PCIe bus to complete transactions out of order. NVIDIA recommends enabling this setting when you use GPUDirect Storage to improve performance.

Run the nvidia-relaxed-ordering-nvme.sh utility.

sudo /bin/nvidia-relaxed-ordering-nvme.sh enable

Note

This feature applies to DGX A100/A800 systems only.

Configuring NVMe Interrupt Coalescing#

The nvidia-nvme-options package, which is installed on all DGX systems, automatically configures NVMe interrupt coalescing on all Samsung, Kioxia, and Micron drives at each boot. To disable this setting or manually configure the setting, issue the following commands:

To disable the setting:

sudo systemctl stop nvidia-nvme-interrupt-coalescing.service
sudo systemctl disable nvidia-nvme-interrupt-coalescing.service

To enable the setting manually:

sudo /usr/bin/nvidia-nvme-interrupt-coalescing.sh enable

To disable the setting manually:

sudo /usr/bin/nvidia-nvme-interrupt-coalescing.sh disable