NVIDIA MLNX_OFED Documentation v24.01-0.3.3.1
NVIDIA MLNX_OFED Documentation v24.01-0.3.3.1

General Support

Supported Operating Systems

Operating System

Architecture

Default Kernel Version (Primary)/

Tested with Kernel Version (Community)

OS Support Model

ASAP2 OVS-Kernel SR-IOV

ASAP2 OVS-DPDK SR-IOV

NFS over RDMA

NVMe

GPUDirect Storage (GDS)

UCX-CUDA Version

Alma 8.5

x86_64

4.18.0-348.12.2.EL8_5.X86_64

Community

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

Anolis OS 8.4

AArch64

4.18.0-348.2.1.AN8_4.AARCH64

Community

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

4.18.0-305.AN8.X86_64

Community

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

Anolis OS 8.6

AArch64

5.10.134+

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

5.10.134+

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

BCLINUX21.10SP2

AArch64

4.18.0-526.el8.aarch64

Primary

error.svg

error.svg

error.svg

check.svg

error.svg

error.svg

x86_64

4.18.0-526.el8.x86_64

Primary

error.svg

error.svg

check.svg

check.svg

error.svg

error.svg

CentOS Stream v8

AArch64

4.18.0-535.el8.aarch64

Community

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

4.18.0-535.el8.x86_64

Community

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

CentOS Stream v9

AArch64

5.14.0-407.el9.x86_64

Community

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

5.14.0-407.el9.aarch64

Community

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

CTYUNOS2.0

AArch64

4.19.90-2102.2.0.0062.ctl2.aarch64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

4.19.90-2102.2.0.0062.ctl2.x86_64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

CTYUNOS23.01

AArch64

5.10.0-136.12.0.86.ctl3.aarch64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

5.10.0-136.12.0.86.ctl3.x86_64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

Debian10.8

AArch64

4.19.0-14-arm64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

Debian10.9

x86_64

4.19.0-14-amd64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

4.19.0-16-amd64

Primary

check.svg

error.svg

error.svg

check.svg

error.svg

error.svg

Debian10.13

AArch64

4.19.0-21-arm64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

4.19.0-21-amd64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

Debian11.3

AArch64

5.10.0-13-arm64

Primary

check.svg

error.svg

error.svg

check.svg

error.svg

error.svg

x86_64

5.10.0-13-amd64

Primary

check.svg

error.svg

error.svg

check.svg

error.svg

error.svg

Debian12

AArch64

6.1.0-10-arm64

Primary

check.svg

error.svg

error.svg

check.svg

error.svg

error.svg

x86_64

6.1.0-10-amd64

Primary

check.svg

error.svg

error.svg

check.svg

error.svg

error.svg

EulerOS2.0sp9

AArch64

4.19.90-vhulk2006.2.0.h171.eulerosv2r9.aarch64

Community

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

4.18.0-147.5.1.0.h269.eulerosv2r9.x86_64

Community

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

EulerOS2.0sp10

AArch64

4.19.90-vhulk2110.1.0.h860.eulerosv2r10.aarch64

Community

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

4.18.0-147.5.2.4.h694.eulerosv2r10.x86_64

Community

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

EulerOS2.0sp11

AArch64

5.10.0-60.18.0.50.h323.eulerosv2r11.aarch64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

5.10.0-60.18.0.50.h323.eulerosv2r11.x86_64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

EulerOS2.0sp12

AArch64

5.10.0-136.12.0.86.h1032.eulerosv2r12.aarch64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

5.10.0-136.12.0.86.h1032.eulerosv2r12.x86_64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

KYLIN10SP2

AArch64

4.19.90-24.4.v2101.ky10.aarch64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

4.19.90-24.4.v2101.ky10.x86_64

Primary

check.svg

error.svg

error.svg

error.svg

error.svg

error.svg

KYLIN10SP3

AArch64

4.19.90-52.22.v2207.ky10.aarch64

Primary

check.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

4.19.90-52.22.v2207.ky10.x86_64

Primary

check.svg

error.svg

error.svg

error.svg

error.svg

error.svg

Mariner 2.0

x86_64

5.15.118.1-1.cm2.x86_64

Community

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

Oracle Linux 7.9

x86_64

5.4.17-2011.6.2.el7uek.x86_64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

Oracle Linux 8.4

x86_64

5.4.17-2102.201.3.el8uek.x86_64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

Oracle Linux 8.6

x86_64

5.4.17-2136.307.3.1.el8uek.x86_64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

Oracle Linux 8.7

x86_64

5.15.0-3.60.5.1.el8uek.x86_64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

Oracle Linux 8.8

x86_64

5.15.0-101.103.2.1.el8uek.x86_64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

Oracle Linux 9.0

x86_64

5.15.0-0.30.19.el9uek.x86_64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

Oracle Linux 9.1

x86_64

5.15.0-3.60.5.1.el9uek.x86_64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

Oracle Linux 9.2

x86_64

5.15.0-101.103.2.1.el9uek.x86_64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

OpenSUSE 15.3

AArch64

-

Community

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

5.3.18-150300.59.43-DEFAULT

Community

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

OPENEULER20.03SP1

AArch64

4.19.90-2012.4.0.0053.OE1.AARCH64

Community

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

4.19.90-2110.8.0.0119.OE1.X86_64

Community

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

OPENEULER20.03SP3

AArch64

4.19.90-2112.8.0.0131.oe1.aarch64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

4.19.90-2112.8.0.0131.oe1.x86_64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

OPENEULER22.03

AArch64

5.10.0-60.18.0.50.oe2203.aarch64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

5.10.0-60.18.0.50.oe2203.x86_64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

Photon OS 3.0

x86_64

4.19.225-3.ph3

Community

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

RHEL/CentOS8.0

AArch64

4.18.0-80.el8.aarch64

Primary

check.svg

check.svg

error.svg

error.svg

check.svg

12.3

x86_64

4.18.0-80.el8.x86_64

Primary

check.svg

check.svg

error.svg

error.svg

check.svg

12.3

RHEL/CentOS8.1

AArch64

4.18.0-147.el8.aarch64

Primary

check.svg

check.svg

error.svg

error.svg

check.svg

12.3

x86_64

4.18.0-147.el8.x86_64

Primary

check.svg

check.svg

error.svg

error.svg

check.svg

12.3

RHEL/CentOS8.2

AArch64

4.18.0-193.el8.aarch64

Primary

check.svg

check.svg

error.svg

check.svg

check.svg

12.3

x86_64

4.18.0-193.el8.x86_64

Primary

check.svg

check.svg

check.svg

check.svg

check.svg

12.3

RHEL/CentOS8.3

AArch64

4.18.0-240.el8.aarch64

Primary

check.svg

check.svg

error.svg

error.svg

check.svg

12.3

x86_64

4.18.0-240.el8.x86_64

Primary

check.svg

check.svg

error.svg

error.svg

check.svg

12.3

RHEL/CentOS8.4

AArch64

4.18.0-305.el8.aarch64

Primary

check.svg

check.svg

error.svg

check.svg

check.svg

12.3

x86_64

4.18.0-305.el8.x86_64

Primary

check.svg

check.svg

check.svg

check.svg

check.svg

12.3

RHEL/CentOS/ Rocky8.5

AArch64

4.18.0-348.el8.aarch64

Primary

check.svg

check.svg

error.svg

check.svg

check.svg

12.3

x86_64

4.18.0-348.el8.x86_64

Primary

check.svg

check.svg

check.svg

check.svg

check.svg

12.3

RHEL/Rocky8.6

AArch64

AArch644.18.0-372.41.1.el8_6.aarch64

Primary

check.svg

check.svg

error.svg

check.svg

check.svg

12.3

x86_64

4.18.0-372.41.1.el8_6.x86_64

Primary

check.svg

check.svg

check.svg

check.svg

check.svg

12.3

RHEL/Rocky8.7

AArch64

4.18.0-425.14.1.el8_7.aarch64

Primary

check.svg

error.svg

error.svg

check.svg

check.svg

12.3

x86_64

4.18.0-425.14.1.el8_7.x86_64

Primary

check.svg

error.svg

check.svg

check.svg

check.svg

12.3

RHEL/Rocky8.8

AArch64

4.18.0-477.10.1.el8_8.aarch64

Primary

check.svg

error.svg

error.svg

check.svg

check.svg

12.3

x86_64

4.18.0-477.10.1.el8_8.x86_64

Primary

check.svg

error.svg

check.svg

check.svg

check.svg

12.3

RHEL/Rocky8.9

AArch64

4.18.0-513.5.1.el8_9.aarch64

Primary

check.svg

error.svg

error.svg

check.svg

check.svg

12.3

x86_64

4.18.0-513.5.1.el8_9.x86_64

Primary

check.svg

error.svg

check.svg

check.svg

check.svg

12.3

RHEL/Rocky9.0

AArch64

5.14.0-70.46.1.el9_0.aarch64

Primary

check.svg

error.svg

error.svg

check.svg

check.svg

12.3

x86_64

5.14.0-70.46.1.el9_0.x86_64

Primary

check.svg

error.svg

check.svg

check.svg

check.svg

12.3

RHEL/Rocky9.1

AArch64

5.14.0-162.19.1.el9_1.aarch64

Primary

check.svg

error.svg

error.svg

check.svg

check.svg

12.3

x86_64

5.14.0-162.19.1.el9_1.x86_64

Primary

check.svg

error.svg

check.svg

check.svg

check.svg

12.3

RHEL/Rocky9.2

AArch64

5.14.0-284.11.1.el9_2.aarch64

Primary

check.svg

error.svg

error.svg

check.svg

check.svg

12.3

x86_64

5.14.0-284.11.1.el9_2.x86_64

Primary

check.svg

error.svg

check.svg

check.svg

check.svg

12.3

RHEL/Rocky9.3

AArch64

5.14.0-362.8.1.el9_3.aarch64

Primary

check.svg

error.svg

check.svg

check.svg

check.svg

12.3

x86_64

5.14.0-362.8.1.el9_3.x86_64

Primary

check.svg

error.svg

check.svg

check.svg

check.svg

12.3

SLES15SP2

AArch64

5.3.18-22-default

Primary

check.svg

check.svg

error.svg

check.svg

error.svg

error.svg

x86_64

5.3.18-22-default

Primary

check.svg

check.svg

check.svg

check.svg

error.svg

error.svg

SLES15SP3

AArch64

5.3.18-57-default

Primary

check.svg

error.svg

error.svg

check.svg

error.svg

error.svg

x86_64

5.3.18-57-default

Primary

check.svg

error.svg

check.svg

check.svg

error.svg

error.svg

SLES15SP4

AArch64

5.14.21-150400.22-default

Primary

check.svg

error.svg

error.svg

check.svg

error.svg

error.svg

x86_64

5.14.21-150400.22-default

Primary

check.svg

error.svg

check.svg

check.svg

error.svg

error.svg

SLES15SP5

AArch64

5.14.21-150500.53-default

Primary

check.svg

error.svg

error.svg

check.svg

error.svg

error.svg

x86_64

5.14.21-150500.53-default

Primary

check.svg

error.svg

check.svg

check.svg

error.svg

error.svg

Ubuntu20.04

AArch64

5.4.0-26-generic

Primary

check.svg

check.svg

error.svg

check.svg

check.svg

12.3

x86_64

5.4.0-26-generic

Primary

check.svg

check.svg

check.svg

check.svg

check.svg

12.3

Ubuntu22.04

AArch64

5.15.0-25-generic

Primary

check.svg

check.svg

check.svg

check.svg

check.svg

12.3

x86_64

5.15.0-25-generic

Primary

check.svg

check.svg

check.svg

check.svg

check.svg

12.3

Ubuntu23.10

x86_64

6.5.0-5-generic

Primary

check.svg

error.svg

check.svg

check.svg

error.svg

error.svg

UOS20.1020

AArch64

4.19.90-2109.1.0.0108.up2.uel20.aarch64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

4.19.90-2109.1.0.0108.up2.uel20.x86_64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

UOS20.1040

AArch64

4.19.0-arm64-server

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

x86_64

4.19.0-server-amd64

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

Citrix XenServer Host8.2

x86_64

4.19.0+1

Primary

error.svg

error.svg

error.svg

error.svg

error.svg

error.svg

Kernel 6.7

AArch64

6.7

Primary

check.svg

error.svg

error.svg

check.svg

error.svg

error.svg

x86_64

6.7

Primary

check.svg

error.svg

check.svg

check.svg

error.svg

error.svg

Warning
  • 32 bit platforms are no longer supported in MLNX_OFED

  • For RPM-based distributions, to install OFED on a different kernel, create a new ISO image using mlnx_add_kernel_support.sh script (see the MLNX_OFED User Manual for instructions)

  • Upgrading MLNX_OFED on a cluster requires upgrading all of its nodes to the newest version as well

  • If using MLNX_OFED 4.9 LTS with MLNX_OFED 5.x with upstream verbs, MLNX_OFED 4.9 must be installed with --upstream-libs flag so the verbs libraries match.

  • A combination of 4.9 LTS default verbs and MOFED 5.x upstream verbs is not supported.

  • All operating systems listed above are fully supported in Paravirtualized and SR-IOV environments with Linux KVM Hypervisor

This section reflects which versions were tested and verified for upgrade and downgrade.

Target Version

Versions Verified for Upgrade/Downgrade

Release Type

Release Date

24.01-0.3.3.1

23.10-1.1.9.0 - DOCA-OFED Profile

GA-LTS-Update

November 2023

23.10-1.1.9.0 - MLNX_OFED

GA-LTS-Update

November 2023

5.8-4.1.5.0

GA-LTS-Update

December 2023

This section reflects which versions were tested and verified for multi-version environments.

Target Version

Verified OFED Version Interoperability

Release Type

Release Date

24.01-0.3.3.1

23.10-1.1.9.0

GA-LTS-Update

November 2023

5.8-4.1.5.0

GA-LTS-Update

December 2023

Supported NIC Firmware Versions

Warning

As of version 5.1, ConnectX-3, ConnectX-3 Pro or Connect-IB adapter cards are no longer supported. To work with a version that supports these adapter cards, please refer to version 4.9 long-term support (LTS).

This current version is tested with the following NVIDIA adapter card firmware versions:

Adapter Card

Bundled Firmware Version

BlueField®-2

24.40.1000

ConnectX-7

28.40.1000

ConnectX-6 Lx

26.40.1000

ConnectX-6 Dx

22.40.1000

ConnectX-6

20.40.1000

ConnectX-5/ConnectX-5 Ex

16.35.3006

BlueField

18.33.1048

ConnectX-4

12.28.2006

ConnectX-4 Lx

14.32.1010

For the official firmware versions, please see https://www.nvidia.com/en-us/networking/ → Support → Support → Firmware Download.

The following are the supported non-Linux Virtual Machines in this current version:

NIC

Windows Virtual Machine Type

Minimal WinOF Version

Protocol

ConnectX-6 family

All Windows server editions

MLNX_WinOF2 2.50

IPoIB, ETH

ConnectX-5 family

MLNX_WinOF2 2.50

IPoIB, ETH

ConnectX-4 Lx

Windows 2016 DC

MLNX_WinOF2 2.50

IB, IPoIB, ETH

ConnectX-4

Windows 2012 R2 DC

MLNX_WinOF2 2.50

IB, IPoIB, ETH

ASAP2 Requirements

  • iproute >= 4.12 (for tc support)

  • Upstream Open vSwitch >= 2.8 for CentOS 7.2 NVIDIA openvswitch

ASAP2-Supported Adapter Cards

  • ConnectX-7

  • ConnectX-6 Lx

  • ConnectX-6 Dx

  • ConnectX-5

  • Lustre 2.15.2

  • Lustre 2.12.9

  • Linux operating system

  • Administrator privileges on your machine(s)

  • Disk Space: 1GB

For the OFED Distribution to compile on your machine, some software packages of your operating system (OS) distribution are required.
To install the additional packages, run the following commands per OS:

Operating System

Required Packages Installation Command

RHEL/Oracle Linux

yum install perl pciutils python gcc-gfortran libxml2-python tcsh libnl.i686 libnl expat glib2 tcl libstdc++ bc tk gtk2 atk cairo numactl pkgconfig ethtool lsof

XenServer

yum install perl pciutils python libxml2-python libnl expat glib2 tcl bc libstdc++ tk pkgconfig ethtool

SLES 15

python ethtool libatk-1_0-0 python2-libxml2-python tcsh lib- stdc++6-devel-gcc7 libgtk-2_0-0 tcl libopenssl1_1 libnl3-200 make libcairo2 expat libmnl0 insserv-compat pciutils lsof lib- glib-2_0-0 pkg-config tk

Ubuntu/Debian

apt-get install perl dpkg autotools-dev autoconf libtool auto- make1.10 automake m4 dkms debhelper tcl tcl8.4 chrpath swig graphviz tcl-dev tcl8.4-dev tk-dev tk8.4-dev bison flex dpatch zlib1g-dev curl libcurl4-gnutls-dev python-libxml2 libvirt-bin libvirt0 libnl-dev libglib2.0-dev libgfortran3 automake m4

pkg-config libnuma logrotate ethtool lsof

The following are the unsupported functionalities/features/NICs in the current version:

  • ConnectX-2 adapter card
  • ConnectX-3 adapter card
  • ConnectX-3 Pro adapter card
  • Connect-IB adapter card
  • Soft-RoCE
  • RDMA experimental verbs library (mlnx_lib)
  • CIFS (Common Internet File System) module installation
  • Relational Database Service (RDS)
  • mthca InfiniBand driver
  • Ethernet IPoIB (eIPoIB)
  • InfiniBand Connected transport service
© Copyright 2023, NVIDIA. Last updated on Apr 15, 2024.