DOCA-Host Installation and DKMS Management Guide

DOCA-Host is a host software stack that provides kernel drivers, user space libraries, and management tools for NVIDIA networking products, including ConnectX and BlueField adapters. It ensures high performance, scalability, and compatibility across various operating systems and applications.

The DOCA-Host stack utilizes DKMS (Dynamic Kernel Module Support) to build and install kernel modules directly from source on the host machine.

The changes in architecture include the following points:

• Source-based build – The DOCA repository no longer contains prebuilt kernel drivers; instead, modules are compiled during installation via DKMS.

• Repository structure – NVIDIA no longer publishes repositories for minor OS releases (e.g., RHEL 9.3). Users must now utilize repositories published per Major OS release (e.g., RHEL 9.x), allowing DKMS to handle kernel differences.

• Module signing – Because modules are built locally, they are not signed by NVIDIA.

DKMS is a framework that enables the automatic rebuilding and reinstallation of Linux kernel modules whenever the host server's kernel version is updated.

DKMS offers the following benefits:

• DKMS creates a trigger that compiles drivers against new kernel headers immediately upon update installation.

• This eliminates the need for specific binaries for every minor OS kernel update or manual source recompilation after system updates.

The following operating systems support the DOCA-Host DKMS implementation:

• RHEL (via doca-all)

• Ubuntu (via doca-all)

• Oracle (via doca-all)

• Debian (via doca-all)

• SLES (via doca-ofed)

This architecture applies to all DOCA-Host profiles that include kernel components:

• doca-roce

• doca-ofed

• doca-networking

• doca-all

Select the repository corresponding to the Major OS version. For example, installations on RHEL 9.4 or RHEL 9.6 must use the RHEL 9 repository.

The following steps outline the installation process using Rocky Linux 9.6 as an example target, utilizing the RHEL 9 repository:

1. If DOCA-Host or MLNX-OFED packages are currently installed, they must be removed prior to installation to prevent conflicts.

   Copy

   Copied!

               
   
               
   host# for f in $(rpm -qa | grep -i doca ) ; do sudo yum -y remove $f; done
   host# sudo /usr/sbin/ofed_uninstall.sh --force
   host# sudo yum autoremove
   host# sudo yum makecache
           

2. You must install DKMS and the kernel headers matching your currently running kernel.

   1. Enable EPEL (if required for dkms availability):

      Copy

      Copied!

                  
      
                  
      sudo dnf install -y epel-release
              

   2. Install build tools and headers:

      Copy

      Copied!

                  
      
                  
      sudo dnf install -y dkms gcc make perl mokutil kernel-devel-$(uname -r) kernel-headers-$(uname -r)
              

3. Download and install the Major Version repository RPM (e.g., RHEL 9):

   Copy

   Copied!

               
   
               
   sudo rpm -Uvh <DOCA_HOST_REPO_RPM_FOR_RHEL9>
   sudo dnf makecache
           

4. Install the desired stack (e.g., doca-all) and the firmware updater:

   Copy

   Copied!

               
   
               
   sudo dnf install -y doca-all mlnx-fw-updater
           

Standard installation packages (RPM/DEB) are signed to verify they originate from NVIDIA.

Kernel modules built via DKMS are not NVIDIA-signed. For systems with Secure Boot enabled, DKMS utilizes a locally generated key pair to sign the built modules.

To ensure the modules load correctly on Secure Boot systems, the public key must be enrolled via the Machine Owner Key (MOK) utility:

1. Import the DKMS public key:

   Copy

   Copied!

               
   
               
   sudo mokutil --import /var/lib/dkms/mok.pub
           

2. Reboot the server:

   Copy

   Copied!

               
   
               
   sudo reboot
           

3. Complete the enrollment process via the MOK menu during the system boot sequence.