Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space ConnectX5VPIOCP2DEV and version 2.3


This section describes how to install and test the Mellanox the NVIDIA OFED for Linux package on a single server with a Mellanox a NVIDIA ConnectX-5 adapter card installed.



A server platform with a ConnectX-5 InfiniBand/VPI adapter card installed

  • MT4119 ConnectX®-5 (VPI, IB, EN) (firmware: fw-ConnectX5)
Required Disk Space for Installation1GB
Device IDFor the latest list of device IDs, please visit the Mellanox the NVIDIA website at
Operating SystemLinux operating system.
For the list of supported operating system distributions and kernels, please refer to the Mellanox OFED NVIDIA OFED Release Notes file.
Installer PrivilegesThe installation requires administrator (root) privileges on the target machine.


Downloading NVIDIA OFED

  1. Verify that the system has a Mellanox a NVIDIA network adapter installed.
    The following example shows a system with an installed Mellanox installed NVIDIA adapter card:

    Code Block
    # lspci -v | grep Mellanox
    86:00.0 Network controller [0207]: Mellanox Technologies MT27620 Family
            Subsystem: Mellanox Technologies Device 0014
    86:00.1 Network controller [0207]: Mellanox Technologies MT27620 Family
            Subsystem: Mellanox Technologies Device 0014
  2. Download the ISO image to your host.
    The image’s name has the format MLNX_OFED_LINUX-<ver>-<OS label><CPU arch>.iso
    You can download and install the latest OpenFabrics Enterprise Distribution (OFED) software package available via the Mellanox the NVIDIA web site at > Products > Software > InfiniBand/VPI Drivers > Linux SW/Drivers > Download..

    1. Scroll down to the Download wizard, and click the Download tab.
    2. Choose your relevant package depending on your host operating system.
    3. Click the desired ISO/tgz package.
    4. To obtain the download link, accept the End User License Agreement (EULA).
  3. Use the md5sum utility to confirm the file integrity of your ISO image. Run the following command and compare the result to the value provided on the download page.

    Code Block
    md5sum MLNX_OFED_LINUX-<ver>-<OS label>.iso


Installing NVIDIA OFED

Installation Script

The installation script, mlnxofedinstall, performs the following:


The installation script removes all previously installed Mellanox installed NVIDIA OFED packages and re-installs from scratch. You will be prompted to acknowledge the deletion of the old packages.

  • If you need to install Mellanox install NVIDIA OFED on an entire (homogeneous) cluster, a common strategy is to mount the ISO image on one of the cluster nodes and then copy it to a shared file system such as NFS. To install on all the cluster nodes, use cluster-aware tools (such as pdsh).
  • If your kernel version does not match with any of the offered pre-built RPMs, you can add your kernel version by using the “” script located inside the MLNX_OFED package.


    On Redhat and SLES distributions with errata kernel installed there is no need to use the script. The regular installation can be performed and weak updates mechanism will create symbolic links to the MLNX_OFED kernel modules.

    The “” script can be executed directly from the mlnxofedinstall script. For further information, please see '--add-kernel-support' option below.


    On Ubuntu and Debian distributions drivers installation use Dynamic Kernel Module Support (DKMS) framework. Thus, the drivers' compilation will take place on the host during MLNX_OFED installation. Therefore, using "" is irrelevant on Ubuntu and Debian distributions.

    The following command will create a MLNX_OFED_LINUX ISO image for RedHat 6.3 under the /tmp directory.

    Code Block
    # ./MLNX_OFED_LINUX-x.x-x-rhel6.3-x86_64/ -m /tmp/MLNX_OFED_LINUX-x.x-x-rhel6.3-x86_64/ --make-tgz
    Note: This program will create MLNX_OFED_LINUX TGZ for rhel6.3 under /tmp directory.
    All Mellanox, OEM, OFED, or Distribution IB packages will be removed.
    Do you want to continue?[y/N]:y
    See log file /tmp/mlnx_ofed_iso.21642.log
    Building OFED RPMs. Please wait...
    Removing OFED RPMs...
    Created /tmp/MLNX_OFED_LINUX-x.x-x-rhel6.3-x86_64-ext.tgz


For your machine to be part of the InfiniBand/VPI fabric, a Subnet Manager must be running on one of the fabric nodes. At this point, Mellanox NVIDIA OFED for Linux has already installed the OpenSM Subnet Manager on your machine.
For the list of installation options, run:


After installation completion, information about the Mellanox the NVIDIA OFED installation, such as prefix, kernel version, and installation parameters can be retrieved by running the command /etc/infiniband/info.

Most of the Mellanox the NVIDIA OFED components can be configured or reconfigured after the installation, by modifying the relevant configuration files. See the relevant chapters in this manual for details.


  • Most of MLNX_OFED packages are installed under the “/usr” directory except for the following packages which are installed under the “/opt” directory:
    • fca and ibutils
  • The kernel modules are installed under
    • /lib/modules/`uname -r`/updates on SLES and Fedora Distributions
    • /lib/modules/`uname -r`/extra/mlnx-ofa_kernel on RHEL and other Red Hat like Distributions
  • The firmware of existing network adapter devices will be updated if the following two conditions are fulfilled:
    • The installation script is run in default mode; that is, without the option ‘--without-fw-update’
    • The firmware version of the adapter device is older than the firmware version included with the Mellanox the NVIDIA OFED ISO image
      Note: If an adapter’s flash was originally programmed with an Expansion ROM image, the automatic firmware update will also burn an Expansion ROM image.
  • In case that your machine has an unsupported network adapter device, no firmware update will occur and the error message below will be printed.
   The firmware for this device is not distributed inside Mellanox driver: 0000:01:00.0 (PSID: IBM2150110033)
To obtain firmware for this device, please contact your HW vendor.


Driver Load Upon System Boot

Upon system boot, the Mellanox the NVIDIA drivers will be loaded automatically.

To prevent automatic load of the Mellanox the NVIDIA drivers upon system boot:


Use the script /usr/sbin/ to uninstall the Mellanox the NVIDIA OFED package. The script is part of the ofed-scripts RPM.


  1. Log into the installation machine as root.
  2. Mount the ISO image on your machine and copy its content to a shared location in your network.

    Code Block
    # mount -o ro,loop MLNX_OFED_LINUX-<ver>-<OS label>-<CPU arch>.iso /mnt
  3. Download and install Mellanox Technologies install NVIDIA GPG-KEY:
    The key can be downloaded via the following link:

    Code Block
    # wget
    --2014-04-20 13:52:30--
    Connecting to||:80... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 1354 (1.3K) [text/plain]
    Saving to: ?RPM-GPG-KEY-Mellanox?
    100%[=================================================>] 1,354 --.-K/s in 0s
    2014-04-20 13:52:30 (247 MB/s) - ?RPM-GPG-KEY-Mellanox? saved [1354/1354]
  4. Install the key.

    Code Block
    # sudo rpm --import RPM-GPG-KEY-Mellanox
    warning: rpmts_HdrFromFdno: Header V3 DSA/SHA1 Signature, key ID 6224c050: NOKEY
    Retrieving key from file:///repos/MLNX_OFED/<MLNX_OFED file>/RPM-GPG-KEY-Mellanox
    Importing GPG key 0x6224C050:
    Userid: "Mellanox Technologies (Mellanox Technologies - Signing Key v2) <>"
    From : /repos/MLNX_OFED/<MLNX_OFED file>/RPM-GPG-KEY-Mellanox
    Is this ok [y/N]:
  5. Check that the key was successfully imported.

    Code Block
    # rpm -q gpg-pubkey --qf '%{NAME}-%{VERSION}-%{RELEASE}\t%{SUMMARY}\n' | grep Mellanox
    gpg-pubkey-a9e4b643-520791ba gpg(Mellanox Technologies <>)
  6. Create a yum repository configuration file called "/etc/yum.repos.d/mlnx_ofed.repo" with the following content:

    Code Block
    name=MLNX_OFED Repository
    baseurl=file:///<path to extracted MLNX_OFED package>/RPMS
    gpgkey=file:///<path to the downloaded key RPM-GPG-KEY-Mellanox>
  7. Check that the repository was successfully added.

    Code Block
    # yum repolist
    Loaded plugins: product-id, security, subscription-manager
    This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
    repo id      repo name                                status
    mlnx_ofed    MLNX_OFED Repository                     108
    rpmforge     RHEL 6Server - - dag        4,597
    repolist: 8,351


Use the script /usr/sbin/ to uninstall the Mellanox the NVIDIA OFED package. The script is part of the ofed-scripts RPM.


  1. Log into the installation machine as root.
  2. Extract the MLNX_OFED pacakge package on a shared location in your network.
    You can download it from > Products > Software> InfiniBand/VPI Drivers.
  3. Create an apt-get repository configuration file called "/etc/apt/sources.list.d/mlnx_ofed.list" with the following content:

    Code Block
    # deb file:/<path to extracted MLNX_OFED package>/DEBS ./
  4. Download and install Mellanox Technologies install NVIDIA GPG-KEY.

    Code Block
    # wget -qO - | sudo apt-key add -
  5. Check that the key was successfully imported.

    Code Block
    # apt-key list
    pub 1024D/A9E4B643 2013-08-11
    uid Mellanox Technologies <>
    sub 1024g/09FCC269 2013-08-11
  6. Update the apt-get cache.

    Code Block
    # sudo apt-get update


  1. View the available package groups by invoking:

    Code Block
    <pre># apt-cache search mlnx-ofed-
    mlnx-ofed-vma-eth - MLNX_OFED vma-eth installer package (with DKMS support)
    mlnx-ofed-hpc - MLNX_OFED hpc installer package (with DKMS support)
    mlnx-ofed-vma-vpi - MLNX_OFED vma-vpi installer package (with DKMS support)
    mlnx-ofed-basic - MLNX_OFED basic installer package (with DKMS support)
    mlnx-ofed-vma - MLNX_OFED vma installer package (with DKMS support)
    mlnx-ofed-all - MLNX_OFED all installer package (with DKMS support)


    Code Block
    mlnx-ofed-all        MLNX_OFED all installer package.
    mlnx-ofed-basic      MLNX_OFED basic installer package.
    mlnx-ofed-vma        MLNX_OFED vma installer package.
    mlnx-ofed-hpc        MLNX_OFED HPC installer package.
    mlnx-ofed-vma-eth    MLNX_OFED vma-eth installer package.
    mlnx-ofed-vma-vpi    MLNX_OFED vma-vpi installer package.
  2. Install the desired group.

    Code Block
    # apt-get install '<group name>'


    Code Block
    # apt-get install mlnx-ofed-all

    Installing MLNX_OFED using the “apt-get” tool does not automatically update the firmware. To update the firmware to the version included in MLNX_OFED package, run: # apt-get install mlnx-fw-updater
    Or, update the firmware to the latest version available on Mellanox Technologies’ Web site the NVIDIA as described in Updating Adapter Firmware.


Use the script /usr/sbin/ to uninstall the Mellanox the NVIDIA OFED package. The script is part of the ofed-scripts package.


To update the device online on the machine from the Mellanox the NVIDIA site, use the following command line:


To update the device manually, please refer to the OEM Firmware Download page at

In case that you ran the mlnxofedinstall script with the ‘--without-fw-update’ option or you are using an OEM card and now you wish to (manually) update firmware on your adapter card(s), you need to perform the steps below. The following steps are also appropriate in case that you wish to burn newer firmware that you have downloaded from Mellanox Technologies’ Web the NVIDIA web site ( > Support > Firmware Download).

  1. Get the device’s PSID.

    Code Block
    mlxfwmanager_pci | grep PSID
    PSID: MT_1210110019
  2. Download the firmware BIN file from the Mellanox the NVIDIA website or the OEM website.

  3. Burn the firmware.

    Code Block
    mlxfwmanager_pci -i <fw_file.bin>
  4. Reboot your machine after the firmware burning is completed.


All kernel modules included in MLNX_OFED for RHEL7 and SLES12 are signed with x.509 key to support loading the modules when Secure Boot is enabled.



NVIDIA's x.509 Public Key on Your Systems

In order to support loading MLNX_OFED drivers when an OS supporting Secure Boot boots on a UEFI-based system with Secure Boot enabled, the Mellanox the NVIDIA x.509 public key should be added to the UEFI Secure Boot key database and loaded onto the system key ring by the kernel.
Follow these steps below to add the MellanoxNVIDIA's x.509 public key to your system:


Prior to adding the Mellanox's the NVIDIA x.509 public key to your system, please make sure that (1) The 'mokutil' package is installed on your system, and (2) The system is booted in UEFI mode.

  1. Download the x.509 public key.

    Code Block
    # wget
  2. Add the public key to the MOK list using the mokutil utility.

    Code Block
    # mokutil --import mlnx_signing_key_pub.der
  3. Reboot the system.


Depending on the application of the user's system, it may be necessary to modify the default configuration of network adapters based on the ConnectX® adapters. In case that tuning is required, please refer to the Performance Tuning Guide for Mellanox NVIDIA Network Adapters at