DOCA-Host Installation and Upgrade

The following options are available for installing or upgrading DOCA-Host:

Full uninstallation is required before installing DOCA-Host:

            

            
host# for f in $( dpkg --list | grep -E 'doca|flexio|dpa-gdbserver|dpa-stats|dpa-resource-mgmt|dpaeumgmt' | awk '{print $2}' ); do echo $f ; sudo apt remove --purge $f -y ; done
host# sudo /usr/sbin/ofed_uninstall.sh --force
host# sudo apt-get autoremove
        

            

            
host# for f in $(rpm -qa | grep -i doca ) ; do sudo yum -y remove $f; done
host# sudo /usr/sbin/ofed_uninstall.sh --force
host# sudo yum autoremove
host# sudo yum makecache
        

1. Install DOCA local repo package for host:

   • For deb-based distributions:

   • 1. Download the DOCA host repo from the NVIDIA DOCA Downloads page.

     2. Install the deb repo:

        Copy

        Copied!

                    
        
                    
        host# sudo dpkg -i <repo_file>.deb
                

     3. Run the update command:

        Copy

        Copied!

                    
        
                    
        host# apt-get update
                

     4. Install any of the profiles supported for your OS and architecture. Refer to "Supported Host OS per DOCA-Host Installation Profile" (e.g., doca-all) for more information.

        Copy

        Copied!

                    
        
                    
        host# sudo apt install -y doca-all
                

     5. Update the firmware:

        Copy

        Copied!

                    
        
                    
        host# sudo apt install -y mlnx-fw-updater
                

   • For RPM-based distributions:

     Note

     DKMS ≥ 3.2 must be installed on the system as a package.

     1. Download the DOCA host repo from the NVIDIA DOCA Downloads page.

     2. Install the rpm repo:

        Copy

        Copied!

                    
        
                    
        host# sudo rpm -Uvh <repo_file>.rpm
                

     3. For non-SLES systems, run:

        Copy

        Copied!

                    
        
                    
        host# sudo yum makecache
                

     4. For SLES systems, run the following to verify the package signing:

        Copy

        Copied!

                    
        
                    
        host# zypper --gpg-auto-import-keys refresh
                

        Note

        Continue the procedure using zypper instead of yum.

     5. Install any of the profiles supported for your OS and architecture (e.g., doca-all):

        Copy

        Copied!

                    
        
                    
        host# yum/zypper install -y doca-all
                

     6. Update the firmware:

        Copy

        Copied!

                    
        
                    
        host# yum/zypper install -y mlnx-fw-updater
                

   Note

   If you are running on a secure-boot-enabled OS, please refer to "Kernel Module Signing" before continuing.

2. Load the drivers:

   Copy

   Copied!

               
   
               
   host# sudo /etc/init.d/openibd restart
           

3. Initialize MST:

   Copy

   Copied!

               
   
               
   host# sudo mst restart
           

To update existing DOCA host packages, follow these steps:

1. Install the newer version of the DOCA host repository as detailed in section "Installing Software on Host".

2. Upgrade the DOCA packages:

   • To upgrade from DOCA version 2.5.x, all DOCA and OFED related packages should be removed . For detailed instructions on how to uninstall these packages, please refer to the uninstall section "Uninstalling Software from Host".

   • To upgrade from DOCA 2.6.0 or later, use the following command:

     • For DEB-based distributions:

       Copy

       Copied!

                   
       
                   
       host# apt install <doca-profile>
               

     • For RPM-based distributions:

       Copy

       Copied!

                   
       
                   
       host# yum upgrade <doca-profile>
               

     • For SLES systems:

       Copy

       Copied!

                   
       
                   
       host# zypper update <doca-profile>
               

   • Before upgrading mlnx-fw-updater, make sure to restart MST first:

     Copy

     Copied!

                 
     
                 
     host# mst restart
             

This section highlights the signing methods used to ensure the authenticity and integrity of the packages and kernel modules.

Package Signing

RPM packages are signed with a GPG key, enabling users to confirm that the packages come from a trusted NVIDIA source and have not been altered.

During installation, package managers automatically fetch and verify the signature while updating their metadata cache.

Kernel Module Signing

With the integration of DKMS (Dynamic Kernel Module Support), DOCA-Host no longer includes pre-built, Nvidia-signed kernel drivers. Instead, the DOCA-Host installation process uses DKMS to automatically build and sign the kernel modules using a locally generated key pair.

To load these drivers on a secure-boot-enabled OS, the DKMS-generated public key must be enrolled in the system's key database. This enrollment is performed after the DOCA-Host installation is complete:

1. Import the public key to the Machine Owner Key (MOK) list using the mokutil utility:

   Copy

   Copied!

               
   
               
   # mokutil --import /var/lib/dkms/mok.pub
           

2. You are prompted to create and confirm a password for this one-time enrollment request.

3. Reboot the system.

4. During the boot process, follow the on-screen prompts from the MOK utility to enroll the key. You will need the password created in step 2.

doca-kernel-support is a script intended to rebuild kernel modules included with DOCA-host. It builds a single package (i.e., rpm or deb) that contains a repository of packages (i.e., dnf or apt).

DOCA-host provides binary builds of kernel modules for some specific kernels. This script rebuilds kernel modules included with DOCA-host for a custom kernel version and creates an RPM or DEB package that holds all of those rebuilt modules for easy install.

To run doca-kernel-support:

1. Run:

   Copy

   Copied!

               
   
               
   host# apt/yum install -y doca-extra
           

2. Execute the doca-kernel-support script which rebuilds and installs the DOCA-host kernel modules with the running kernel:

   Copy

   Copied!

               
   
               
   host# /opt/mellanox/doca/tools/doca-kernel-support
           

   Note

   To sign the kernel modules, run this command instead:

   Copy

   Copied!

               
   
               
   host# MODULE_SIGN_PUB_KEY=/path/to/pub_key.der MODULE_SIGN_PRIV_KEY=/path/to/priv_key.priv WITH_MOD_SIGN=1 /opt/mellanox/doca/tools/doca-kernel-support
           

   The output may end with rpm or deb with the following general format:

   Copy

   Copied!

               
   
               
   doca-kernel-support: Built single package: <doca-kernel-repo>
   doca-kernel-support: Done
           

   Output example:

   Copy

   Copied!

               
   
               
   doca-kernel-support: Built single package: /tmp/DOCA.EuUfkWfV7Z/doca-kernel-repo-2.9.0-1.kver.5.14.0.356.el9.x86.64.x86_64.rpm
   doca-kernel-support: Done
           

   Note

   doca-kernel-support does not support customized or unofficial kernels.

3. Install the generated meta-package on the desired DOCA profile:

   Info

   The examples provided install DOCA kernel repos (RPM/DEB) with doca-ofed profile, but other profiles may be installed if they are supported.

   • For RPM-based distributions:

     Copy

     Copied!

                 
     
                 
     host# rpm -Uvh <doca-kernel-repo>
     host# dnf makecache
     host# dnf install <doca-userspace-metapackage>   
     host# dnf install --disablerepo=doca doca-kernel-KERNEL_VERSION     # For all the meta-package, not just this single package					
             

     Example:

     Copy

     Copied!

                 
     
                 
     host# rpm -Uvh /tmp/DOCA.EuUfkWfV7Z/doca-kernel-repo-2.9.0-1.kver.5.14.0.356.el9.x86.64.x86_64.rpm
     host# dnf makecache
     host# dnf install doca-ofed-userspace
     host# dnf install --disablerepo=doca doca-kernel-5.14.0.356.el9.x86.64.x86_64.noarch
             

   • For DEB-based distributions:

     Copy

     Copied!

                 
     
                 
     host# dpkg -i <doca-kernel-repo>
     host# apt update
     host# apt install <doca-profile>
             

     Example:

     Copy

     Copied!

                 
     
                 
     host# dpkg -i /tmp/DOCA.J8JYxEmffD/doca-kernel-repo-2.9.0-6.4.0.mlnx_2.9.0_amd64.deb
     host# apt update
     host# apt install doca-ofed
             

4. (Optional) Retrieve installed packages and their versions as part of DOCA Host installation:

   Collapse Source

   Copy

   Copied!

               
   
               
   host# /opt/mellanox/doca/tools/doca-info
    
   Versions:
   - DOCA Base MLNX_OFED_LINUX-24.07-0.5.5.0
   - MFT 4.29.0-127
    
   UEFI\ATF versions:
   - mst_device: mt41692_pciconf0
        UEFI Version: 4.7.0-42-g13081ae
        ATF Version: 4.7.0-25-g5569834
    
   Firmware (Current):
   - BlueField-3 32.41.1000
    
   DOCA:
   - doca-all 2.8.0-0.0.4
   - doca-apsh-config 2.8.0079-1
   - doca-bench 2.8.0079-1
   …
    
   DOCA Dependencies:
   …
   - flexio 24.07.2300
   - mlnx-dpdk 22.11.0-2407.0.10
    
    
   OFED:
   …
   - rdma-core 2407mlnx52-1.2407055
   …
   - ucx 1.17.0-1.2407055
   …
           

   Note

   If BlueField has a BF-Bundle version older than 2.7.0 installed on it, the UEFI/ATF versions would appear as N/A. If your version is 2.7.0 or higher and still see N/A, then perform driver restart on the host:

   Copy

   Copied!

               
   
               
   /etc/init.d/openibd restart
           

1. Before installation, search for the required package using your system's package manager:

   Copy

   Copied!

               
   
               
   # apt/zypper/dnf/yum search mlnx-nvme
   # apt/zypper/dnf/yum search mlnx-nfsrdma
           

2. Once the correct package name is identified, install it using:

   Copy

   Copied!

               
   
               
   # apt/zypper/dnf/yum install <full-package-name>
           

Installation examples:

• For SLES:

  Copy

  Copied!

              
  
              
  # zypper search mlnx-nvme
  # zypper install mlnx-nvme-kmp-default
          

• For Debian-based systems:

  Copy

  Copied!

              
  
              
  # apt search mlnx-nvme
  # apt search mlnx-nfsrdma
  # apt install mlnx-nvme-dkms
  # apt install mlnx-nfsrdma-dkms
          

VMA Installation

VMA is compatible with the latest DOCA-Host drivers, firmware, and libraries.

For users looking to install or upgrade VMA, DOCA provides doca-libvma, a meta-package that installs the driver stack, libvma, and all necessary dependencies.

To install doca-libvma, follow the instructions under section "Installing Software on Host" and substitute doca-all with doca-libvma in the following command:

            

            
# Original command
sudo <apt/yum/zypper> install -y doca-all
# Updated command
sudo <apt/yum/zypper> install -y doca-libvma
        

For the list of systems required to work with VMA, refer to the VMA documentation.

XLIO Installation

XLIO is compatible with the latest DOCA-Host drivers, firmware, and libraries.

For users looking to install or upgrade XLIO, DOCA doca-libxlio, a meta-package that installs the driver stack, libxlio, and all necessary dependencies.

To install doca-libxlio, follow the instructions under section "Installing Software on Host" while substituting doca-all with doca-libxlio in the following command:

            

            
# Original command
sudo <apt/yum/zypper> install -y doca-all
# Updated command
sudo <apt/yum/zypper> install -y doca-libxlio
        

For the list of systems required to work with XLIO, please see the XLIO documentation.

The installation procedure does not install proprietary packages. Those packages are installed upon request.

List of close-source proprietary packages:

• Clusterkit

• DPCP

• hcoll

• sharp

• ibutils2

• opensm

Currently, the only way to install these packages is by using an already-built RPM or DEB file from a similar primary OS.

The following table maps community OSs which are most similar to primary OSs:

Download and Install MFT and Firmware

• Download MFT

• Download firmware