Appendix A: Installing Software

The Ubuntu Server uses the apt package manager to install, update, and remove packages. The utility can also be used to manage repositories. For more information about using apt, refer to Ubuntu Package Management.

A.1 NVIDIA DOCA-OFED

NVIDIA MLNX_OFED has transitioned to DOCA-OFED. Refer to the NVIDIA MLNX OFED to DOCA-OFED Transition Guide for information about this transition. Refer to the NVIDIA DOCA Installation Guide for Linux for more information.

To install NVIDIA DOCA-OFED on Ubuntu:

1. Add the NVIDIA DOCA-OFED repository to your system.

   • On Ubuntu 22.04:

     wget -qO - https://linux.mellanox.com/public/repo/doca/GPG-KEY-Mellanox.pub
     | sudo gpg --dearmor -o /usr/share/keyrings/GPG-KEY-Mellanox.pub
     echo "deb [signed-by=/usr/share/keyrings/GPG-KEY-Mellanox.pub]
     https://linux.mellanox.com/public/repo/doca/latest/ubuntu22.04/arm64-sbsa /" | sudo tee /etc/apt/sources.list.d/doca.list
     sudo apt-get update
     

   • On Ubuntu 24.04:

     wget -qO - https://linux.mellanox.com/public/repo/doca/GPG-KEY-Mellanox.pub |
     sudo gpg --dearmor -o /usr/share/keyrings/GPG-KEY-Mellanox.pub
     sudo dd status=none of=/etc/apt/sources.list.d/doca.sources << EOF
     Types: deb
     URIs: https://linux.mellanox.com/public/repo/doca/latest/ubuntu24.04/arm64-sbsa
     Suites: /
     Signed-By: /usr/share/keyrings/GPG-KEY-Mellanox.pub
     EOF
     sudo apt-get update
     

     Note

     This sequence configures the repository for the latest available version of NVIDIA DOCA-OFED. If you need a specific version, replace <latest> with the version. For example, to get the latest DOCA-OFED 2.9 version for Ubuntu 24.04, use the https://linux.mellanox.com/public/repo/doca/2.9.0/ubuntu24.04/arm64-sbsa/ URI.

2. Install the NVIDIA DOCA software.

   sudo apt-get install doca-ofed -y
   

   For GPUDirect Storage (GDS) functionality, NVIDIA recommends that you install mlnx-nvme-dkms and mlnx-nfsrdma-dkms.

3. When using the NVIDIA BlueField-3 SoC Management Interface, enable the Rshim service.

   sudo systemctl daemon-reload
   sudo systemctl enable rshim
   sudo systemctl start rshim
   

4. To confirm that the NVIDIA BlueField-3 SoC Management Interface is on the system, run the following commands.

   # Print the PCI BDF for the BlueField-3 Soc Management Interface
   sudo lspci | grep "BlueField-3 SoC Management Interface" | awk '{print $1}'
   0006:03:00.2
   0016:03:00.2
   

5. Update the device firmware with the version in the DOCA package.

   sudo apt-get install mlnx-fw-updater
   

6. If the NVIDIA BlueField-3 SoC Management Interface is on the system, install the BF3 bundle.

   sudo bfb-install -rshim rshim<N> -bfb <image_path.bfb>
   

   Where <N> is the rshim device identifier.

7. Reboot the system.

   sudo reboot now
   

A.2 NVIDIA GPU Driver and CUDA Toolkit

Refer to the NVIDIA CUDA Installation Guide for Linux for information about installing the NVIDIA GPU driver and CUDA support for Ubuntu. The minimum driver version for Hopper is GPUR535.129.03.

To install the minimum versions for the Hopper GPU, run the following commands:

sudo apt-get install linux-headers-$(uname -r)
wget
https://developer.download.nvidia.com/compute/cuda/repos/ubuntu$(lsb_release
-rs | sed 's/.//')/sbsa/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring*.deb
sudo apt-get update
sudo apt-get install cuda-toolkit-12-8 -y
sudo apt-get install nvidia-open-570 -y
sudo systemctl enable nvidia-persistenced
sudo reboot now

Note

The open-source GPU driver is required for Hopper GPUs.