Versions Compared

Old Version 8

changes.mady.by.user Boris Kovalev

Saved on

compared with

New Version Current

changes.mady.by.user Boris Kovalev

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Created on Jun 4, 2019

Updated on Sep 13, 2021

On This Page

Introduction

This is a How To that shows how to install and test the NVIDIA ConnectX-5/ConnectX-6 NATIVE ESXi Driver for VMware vSphere ESXi 6.7/7.0 on single host, and perform basic initial configuration steps to enable the driver by using ESXi cli interface.

References

Hardware and Software Requirements

1. A server platform with an adapter card based on one of the following NVIDIA ConnectX®-4/5 PCI Express Adapter Cards (Ethernet, VPI) devices.

2. Installer Privileges: The installation requires administrator privileges on the target machine.

3. Device ID: For the latest list of device IDs, please visit NVIDIA website.

4. Supported NICs / Firmware: Recommended firmware versions you can find here.

Driver Installation (CLI)

1. Enable SSH Access to ESXi server.

2. Log into ESXi vSphere Command-Line Interface with root permissions.

3. Verify that the host is equipped with NVIDIA ConnectX (Mellanox) adapter.


Code Block
languagetext
themeFadeToGrey
titleESXi Console
~ lspci | grep Mellanox

0000:39:00.0 Ethernet controller: Mellanox Technologies ConnectX-6 Dx EN NIC; 100GbE; dual-port QSFP56; PCIe4.0 x16; (MCX623106AC-CDA) [vmnic0]
0000:39:00.1 Ethernet controller: Mellanox Technologies ConnectX-6 Dx EN NIC; 100GbE; dual-port QSFP56; PCIe4.0 x16; (MCX623106AC-CDA) [vmnic1]
Info
Note: in this case, the Mellanox card is using vmnic0 and vmnic1.

4. Verify the driver's version was installed.

ESXi 7.0

Code Block
languagetext
themeFadeToGrey
titleESXi Console
~ esxcli software vib list | grep nmlx
nmlx5-core                     4.1921.1671.10-1vmw-6501-1OEM.702.0.0-8277601 VMW.17473468      MEL     VMwareCertified 2018  2021-0406-24 07
nmlx5-rdma 4.19.16.10-1vmw-650.0.0-8277601 VMW VMwareCertified 2018-04-24rdma                     4.21.71.1-1OEM.702.0.0.17473468      MEL     VMwareCertified   2021-06-07

ESXi 6.7

Code Block
languagetext
themeFadeToGrey
titleESXi Console
~ esxcli software vib list | grep nmlx 

nmlx5-core 4.17.9.12-1vmw.670.0.0.8169922 VMW VMwareCertified 2018-04-25
nmlx5-rdma 4.17.9.12-1vmw.670.0.0.8169922 VMW VMwareCertified 2018-04-25


5. Download a latest Mellanox native ESXi drivers from here.

6. Unzip the binary image (.zip file).

7. Use SCP or any other file transfer method to copy the driver to the required ESXi host.

8. Place a Host in Maintenance Mode the ESXi host.

9. Install the driver.

~ esxcli software vib install –d <path>/<bundle_file>

ESXi 7.0

Code Block
languagetext
themeFadeToGrey
titleESXi Console
~ esxcli software vib install -d /tmp/Mellanox-nmlx5_4.21.71.101-1OEM.702.0.0.17630552.zip

ESXi 6.7

Code Block
languagetext
themeFadeToGrey
titleESXi Console
~ esxcli software vib install -d /tmp/Mellanox-nmlx5_4.17.71.1-1OEM.670.0.0.8169922.zip


10. Reboot the server.

Code Block
languagetext
themeFadeToGrey
titleESXi Console
~ reboot

11. Verify the driver modules was installed and loaded successfully.

Code Block
languagetext
themeFadeToGrey
titleESXi Console
esxcli software vib list | grep nmlx
nmlx5-core                     4.1721.71.1101-1OEM.670702.0.0.816992217630552    MEL     VMwareCertified   20182021-0609-0730
nmlx5-rdma                     4.1721.71.1101-1OEM.670702.0.0.816992217630552    MEL     VMwareCertified 2018  2021-06-07

...09-30
Code Block
languagetext
themeFadeToGrey
titleESXi Console
~ esxcli system module list | grep nmlx5

nmlx5_core                          true        true
nmlx5_rdma                          true        true

12. Check physical network interface status.

Code Block
languagetext
themeFadeToGrey
titleESXi Console
~ esxcli network nic list


Name     PCI Device    Driver      Admin Status  Link Status   Speed  Duplex  MAC Address         MTU  Description
-------  ------------  ----------  ------------  -----------  ------  ------  -----------------  ---- --------------------------- -------------------------
...
vmnic2
vmnic0   0000:0239:00.0  nmlx5_core  Up            Up           100000  Full ec:0d:9a:8c:c7:96   0c:42:a1:24:04:ea  1500  Mellanox Technologies MT28800 Family [ConnectX-5 Ex]
vmnic3 0000:02 ConnectX-6 Dx EN NIC; 100GbE; dual-port QSFP56; PCIe4.0 x16; (MCX623106AC-CDA)
vmnic1   0000:39:00.1  nmlx5_core  Up            Down              0  Half  ec  0c:0d42:9aa1:8c24:c704:97eb  1500  Mellanox Technologies MT28800 Family [ConnectX-5 Ex] ConnectX-6 Dx EN NIC; 100GbE; dual-port QSFP56; PCIe4.0 x16; (MCX623106AC-CDA)

13. Exit Maintenance Mode the ESXi host.


Done!

Related Documents

Authors

Content by LabelshowLabelsfalseshowSpacefalsesortcreationcqllabel in ("esxi","connectx")

Include Page
SA:Boris Kovalev
SA:Boris Kovalev