NVIDIA MLNX_EN Documentation v23.04-1.1.3.0
NVIDIA Docs Hub  NVIDIA Networking  Networking Software  NVIDIA MLNX_EN Documentation v23.04-1.1.3.0  Installing MLNX_EN
NVIDIA MLNX_EN Documentation v23.04-1.1.3.0

On This Page

Installing MLNX_EN

Installation Script

The install installation script performs the following:

  • Discovers the currently installed kernel

  • Uninstalls any previously installed MLNX_EN package

  • Installs the MLNX_EN binary (if they are available for the current kernel)

  • Identifies the currently installed Ethernet network adapters and automatically upgrades the firmware

Warning

If the driver detects unsupported cards on your system, it will abort the installation procedure. To avoid this, make sure to add --skip-unsupported-devices-check flag during installation.

Installation Modes

mlnx_en installer supports 2 modes of installation. The install script selects the mode of driver installation depending on the running OS/kernel version.

  • Kernel Module Packaging (KMP) mode, where the source rpm is rebuilt for each installed flavor of the kernel. This mode is used for RedHat and SUSE distributions.

  • Non KMP installation mode, where the sources are rebuilt with the running kernel. This mode is used for vanilla kernels.

  • By default, the package will install drivers supporting Ethernet only. In addition, the package will include the following new installation options:

    • Full VMA support which can be installed using the installation option “–vma”.

    • Infrastructure to run DPDK using the installation option “–dpdk”.

      Notes:

      • DPDK itself is not included in the package. Users would still need to install DPDK separately after the MLNX_EN installation is completed.

      • Installing VMA or DPDK infrastructure will allow users to run RoCE.

Installation Results

  • For Ethernet only installation mode:

    • The kernel modules are installed under:

      • /lib/modules/`uname -r`/updates on SLES and Fedora Distributions

      • /lib/modules/`uname -r`/extra/mlnx-en on RHEL and other RedHat like Distributions

      • /lib/modules/`uname -r`/updates/dkms/ on Ubuntu

    • The kernel module sources are placed under:
      /usr/src/mlnx-en-<ver>/

  • For VPI installation mode:

    • The kernel modules are installed under:

      • /lib/modules/`uname -r`/updates on SLES and Fedora Distributions

      • /lib/modules/`uname -r`/extra/mlnx-ofa_kernel on RHEL and other RedHat like Distributions

      • /lib/modules/`uname -r`/updates/dkms/ on Ubuntu

    • The kernel module sources are placed under:
      /usr/src/ofa_kernel-<ver>/

Installation Procedure

This section describes the installation procedure of MLNX_EN on NVIDIA adapter cards.

  1. Log into the installation machine as root.

  2. Mount the ISO image on your machine.

    Copy
    Copied!
                
    
            
    host1# mount -o ro,loop mlnx-en-<ver>-<OS label>-<CPU arch>.iso /mnt

  3. Run the installation script.

    Copy
    Copied!
                
    
            
    /mnt/install

  4. Case A: If the installation script has performed a firmware update on your network adapter, you need to either restart the driver or reboot your system before the firmware update can take effect. Refer to the table below to find the appropriate action for your specific card.

    Action \ Adapter

    Driver Restart

    Standard Reboot (Soft Reset)

    Cold Reboot (Hard Reset)

    Standard ConnectX-4/ConnectX-4 Lx or higher

    -

    +

    -

    Adapters with Multi-Host Support

    -

    -

    +

    Socket Direct Cards

    -

    -

    +

    Case B: If the installations script has not performed a firmware upgrade on your network adapter, restart the driver by running: # /etc/init.d/mlnx-en.d restart

The result is a new net-device appearing in the 'ifconfig -a' output.

Additional Installation Procedures

Installing MLNX_EN Using YUM

This type of installation is applicable to RedHat/OL, Fedora, XenServer operating systems.

Setting up MLNX_EN YUM Repository

The package consists of two folders that can be set up as a repository:

  • “RPMS_ETH” - provides the Ethernet only installation mode

  • “RPMS” - provides the RDMA support installation mode

  1. Log into the installation machine as root.

  2. Mount the ISO image on your machine and copy its content to a shared location in your network.

    Copy
    Copied!
                
    
            
    # mount -o ro,loop mlnx-en-<ver>-<OS label>-<CPU arch>.iso /mnt

    You can download the image from nvidia.com/en-us/networking/.com → Products → Software → Ethernet Drivers.

  3. Download and install the NVIDIA GPG-KEY:
    The key can be downloaded via the following link: http://www.mellanox.com/downloads/ofed/RPM-GPG-KEY-Mellanox

    Example:

    Copy
    Copied!
                
    
            
    # wget http://www.mellanox.com/downloads/ofed/RPM-GPG-KEY-Mellanox
--2014-04-20 13:52:30-- http://www.mellanox.com/downloads/ofed/RPM-GPG-KEY-Mellanox
Resolving www.mellanox.com... 72.3.194.0
Connecting to www.mellanox.com|72.3.194.0|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1354 (1.3K) [text/plain]
Saving to: ?RPM-GPG-KEY-Mellanox?
100%[=================================================>] 1,354 --.-K/s in 0s
2014-04-20 13:52:30 (247 MB/s) - ?RPM-GPG-KEY-Mellanox? saved [1354/1354]

  4. Install the key.

    Example:

    Copy
    Copied!
                
    
            
    # sudo rpm --import RPM-GPG-KEY-Mellanox
warning: rpmts_HdrFromFdno: Header V3 DSA/SHA1 Signature, key ID 6224c050: NOKEY
Importing GPG key 0x6224C050:
Userid: "Mellanox Technologies (Mellanox Technologies - Signing Key v2) <support@mellanox.com>"
From : /repos/MLNX_EN/RPM-GPG-KEY-Mellanox
Is this ok [y/N]:

  5. Check that the key was successfully imported.

    Example:

    Copy
    Copied!
                
    
            
    # rpm -q gpg-pubkey --qf '%{NAME}-%{VERSION}-%{RELEASE}\t%{SUMMARY}\n' | grep Mellanox
gpg-pubkey-a9e4b643-520791ba gpg(Mellanox Technologies <support@mellanox.com>)
Rev 3.30
Mellanox Technologies 45

  6. Create a YUM repository configuration file called “/etc/yum.repos.d/mlnx_en.repo” with the following content:

    Copy
    Copied!
                
    
            
    [mlnx_en]
name=MLNX_EN Repository
baseurl=file:///<path to extracted MLNX_EN package>/<RPMS FOLDER NAME>
enabled=1
gpgkey=file:///<path to the downloaded key RPM-GPG-KEY-Mellanox>
gpgcheck=1

    Warning

    Replace <RPMS FOLDER NAME> with “RPMS_ETH” or “RPMS” depending on the desired installation mode (Ethernet only or RDMA).

  7. Check that the repository was successfully added.

    Copy
    Copied!
                
    
            
    # yum repolist
Loaded plugins: product-id, security, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
repo id                                                          repo name                                                                                    status
mlnx_en                                                          MLNX_EN Repository

Installing MLNX_EN Using the YUM Tool

After setting up the YUM repository for MLNX_EN package, install one of the following metadata packages:

  • In case you set up the “RPMS_ETH” folder as the repository (for Ethernet only mode), install:

    Copy
    Copied!
                
    
            
    # yum install mlnx-en-eth-only

  • In case you set up the “RPMS” folder as the repository (for RDMA mode), install either:

    Copy
    Copied!
                
    
            
    # yum install mlnx-en-vma

    Or

    Copy
    Copied!
                
    
            
    # yum install mlnx-en-dpdk

Please note the following:

MLNX_EN provides kernel module RPM packages with KMP support for RHEL and SLES. For other operating systems, kernel module RPM packages are provided only for the operating system’s default kernel. In this case, the group RPM packages have the supported kernel version in their packages name.
If you have an operating systems different than RHEL or SLES, or you have installed a kernel that is not supported by default in MLNX_EN, you can use the mlnx_add_kernel_support.sh script to build MLNX_EN for your kernel.
The script will automatically build the matching group RPM packages for your kernel so that you can still install MLNX_EN via YUM.
Please note that the resulting MLNX_EN repository will contain unsigned RPMs. Therefore, you should set 'gpgcheck=0' in the repository configuration file.
Installing MLNX_EN using the YUM tool does not automatically update the firmware.
To update the firmware to the version included in MLNX_EN package, you can either:

  1. Run:

    Copy
    Copied!
                
    
            
    # yum install mlnx-fw-updater

    OR

  2. Update the firmware to the latest version available on NVIDIA website as described in Updating Firmware After Installation section.

Installing MLNX_EN Using apt-get

This type of installation is applicable to Debian and Ubuntu operating systems.

Setting up MLNX_EN apt-get Repository

The package consists of two folders that can be set up as a repository:

  • “DEBS_ETH” - provides the Ethernet only installation mode.

  • “RPMS” - provides the RDMA support installation mode.

  1. Log into the installation machine as root.

  2. Extract the MLNX_EN package on a shared location in your network.
    You can download it from nvidia.com/en-us/networking/.com → Products → Software → Ethernet Drivers.

  3. Create an apt-get repository configuration file called "/etc/apt/sources.list.d/mlnx_en.list" with the following content:

    Copy
    Copied!
                
    
            
    deb file:/<path to extracted MLNX_EN package>/<DEBS FOLDER NAME> ./

    Warning

    Replace <DEBS FOLDER NAME> with “DEBS_ETH” or “DEBS” depending on the desired installation mode (Ethernet only or RDMA).

  4. Download and install the NVIDIA GPG-KEY.

    Example:

    Copy
    Copied!
                
    
            
    # wget -qO - http://www.mellanox.com/downloads/ofed/RPM-GPG-KEY-Mellanox | sudo apt-key add -

  5. Verify that the key was successfully imported.

    Example:

    Copy
    Copied!
                
    
            
    # apt-key list
pub 1024D/A9E4B643 2013-08-11
uid Mellanox Technologies <support@mellanox.com>
sub 1024g/09FCC269 2013-08-11

  6. Update the apt-get cache.

    Copy
    Copied!
                
    
            
    # sudo apt-get update

Installing MLNX_EN Using the apt-get Tool

After setting up the apt-get repository for MLNX_EN package, install one of the following metadata packages:

  • In case you set up the “DEBS_ETH” folder as the repository (for Ethernet only mode), install:

    Copy
    Copied!
                
    
            
    # apt-get install mlnx-en-eth-only

  • In case you set up the “DEBS” folder as the repository (for RDMA mode), install either:

    Copy
    Copied!
                
    
            
    # apt-get install mlnx-en-vma

    OR

    Copy
    Copied!
                
    
            
    # apt-get install mlnx-en-dpdk

Installing MLNX_EN using the apt-get tool does not automatically update the firmware. To update the firmware to the version included in MLNX_EN package, you can either:

  1. Run:

    Copy
    Copied!
                
    
            
    # apt-get install mlnx-fw-updater

    Or

  2. Update the firmware to the latest version available on NVIDIA website as described in Updating Firmware After Installation section.

Installation Using Repositories

NVIDIA Legacy Libraries can also be installed using the operating system's standard package manager (yum, apt-get, etc.).

  • For RPM based operating systems, follow the steps in “Installing MLNX_EN Using YUM” section, using the directory “MLNX_LIBS" instead of “UPSTREAM_LIBS” when creating the “/etc/yum.repos.d/mlnx_ofed.repo” file.

  • For Debian based operating systems, follow the steps in “Installing MLNX_EN Using apt-get” section using the directory “MLNX_LIBS” instead of “UPSTREAM_LIBS” when creating the “/etc/apt/sources.list.d/mlnx_ofed.list” file.

Finally, for both RPM and Debian based OSs, install the new metadata package called “mlnx-ofed-dpdk-upstream-libs”, which will install both the user space and kernel packages.
If you wish to install only the user space packages, make sure to install the metadata package called “mlnx-ofed-dpdk-upstream-libs-user-only”.

© Copyright 2023, NVIDIA. Last updated on Sep 9, 2023.