Installing VMA
Before you begin, verify you are using a supported operating system and a supported CPU architecture for your operating system. See supported combinations listed in System Requirements and Interoperability.
The current version of VMA is compatible with DOCA-host and offers two profile driver stacks: the doca-libvma profile, which supports Ethernet, and the lighter doca-roce profile, which also supports Ethernet.
The VMA library is delivered as a user-space library, and is called libvma.so.X.Y.Z.
VMA can be installed using one of the following methods (detailed in VMA Installation Options):
As part of DOCA-host:
Installing the combined driver stack and VMA using the doca-libvma Profile
Installing the driver stack only using the doca-roce Profile, then install the VMA library separately using the libvma Profile
Manually:
Building VMA From Sources Tree
Installing VMA via Dedicated Packages
For VMA installation on RHEL 7.x Inbox, see Installing VMA in RHEL 7.x Inbox section.
After completing the installation, verify its success as described in Using sockperf with VMA.
Installing VMA as Part of DOCA-Host
VMAis integrated into the DOCA-Host (doca-libVMA/doca-roce). Therefore, it is recommended to install VMA automatically when installing the NVIDIA DOCA-Host drivers as it depends on drivers’ latest firmware, libraries, and kernel modules. This installation assures VMA’s correct functionality. Since the DOCA-Host has a plethora of distributions for RHEL, Ubuntu and others, you will have to correctly select the DOCA-Host version that matches your distribution.
This option suits users that wish to install a new VMA version or upgrade to the latest VMA version by overriding the previous one. For further information, see NVIDIA DOCA Installation Guide for Linux section in DOCA documentation.
Installing driver stack & VMA as doca-libvma Profile
For VMA, there is a dedicated DOCA profile called doca-libvma, which installs the driver stack & libvma along with all the necessary dependencies.
Refer to the NVIDIA DOCA Installation Guide for Linux section in DOCA documentation and follow the steps for installing doca-all, but use doca-libvma instead. For example:
“sudo <your-package-manager> install -y doca-libvma"
Installing VMA as libvma Profile
Use the DOCA profile called libvma, which includes only the VMA library and its user space dependencies (without driver stack).
For containerized environments, Containers utilize the host system’s kernel, which already provides the required NVIDIA kernel modules and drivers.
Refer to the NVIDIA DOCA Installation Guide for Linux section in DOCA documentation and follow the steps for installing doca-all, but use libvma instead. For example:
“sudo <your-package-manager> install -y libvma dpcp"
Note: libvma-utils, libvma-dev/devel, and sockperf are optional.
For RPM packages:
#sudo <your-
package-manager> install -y sockperf libvma-utils libvma-develFor DEB packages:
#sudo <your-
package-manager> install -y sockperf libvma-utils libvma-dev
Installing driver stack as doca-roce Profile
Users looking for a lighter driver stack should install the doca-roce profile and add the libvma-related packages as needed.
Follow the NVIDIA DOCA Installation Guide for Linux section in DOCA documentation to install doca-roce, then add the relevant libvma packages. For example:
#sudo <your-package-manager> install -y doca-roce
Building VMA From Sources
VMA as an open source enables users to try the code, inspect and modify. This option assumes that VMA (binary package) is already installed on top of NVIDIA driver, and functions correctly (as explained in Installing the VMA Binary Package). Users can download the VMA sources from libvma GitHub and build a new VMA version. This option is suitable for users who wish to implement custom VMA modifications.
Please visit the libvma wiki page for various “how-to” issues.
For building VMA from sources, please visit the following page: https://github.com/Mellanox/libvma/wiki/Build-Instruction
For adding high debug log level to VMA compile it with:
./configure --enable-opt-log=none <other configure parameters>
For libvma configuration file please see here.
Verifying VMA Compilation
The compiled VMA library is located under <path-to-vma-sources-root-tree>/src/vma/.libs/libvma.so
When running a user application, you must set the compiled library into the env variable LD_PRELOAD.
Example:
#LD_PRELOAD=<path-to-vma-sources-root-tree>/src/vma/.libs/libvma.so sockperf ping-pong -t 5 -i 224.22.22.22
As indicated in Running VMA, the appearance of the VMA header verifies that the compiled VMA library is loaded with your application.
Remark: it is recommended to keep the original libvma.so under the distribution’s original location (e.g. /usr/lib64/libvma.so) and not to override it with the newly complied libvma.so.
Installing VMA via Dedicated Packages
VMA can be installed from a dedicated VMA RPM or a Debian package. In this case, please make sure that NVIDIA's driver stack is already installed and that the driver stack and VMA versions match so that VMA functions correctly.
This option is suitable for users who receive an OEM VMA version, or a Fast Update Release VMA version which is newer than the installed VMA version.
This section addresses both RPM and DEB (Ubuntu OS) installations.
VMA includes the following packages which should be saved on your local drive:
The libvma package contains the binary library shared object file (.so), configuration and documentation files
The libvma-utils package contains utilities such as vma_stats to monitor vma traffic and statistics
The libvma-devel (RPM) or libvma-dev (DEB) packages contain VMA’s extended API header files, for extra functionality not provided by the socket API
Before you begin, please verify the following prerequisites:
Check whether VMA is installed:
For RPM packages, run:
#rpm -qil libvma
For DEB packages, run:
#dpkg –s libvma
If the VMA packages are not installed, an appropriate message is displayed.
If the VMA packages are installed, the RPM or the DEB logs the VMA package information and the installed file list.
To uninstall the current VMA:
For RPM packages, run:
#rpm -e libvma
For DEB packages:
#apt-get remove libvma
Installing the VMA Binary Package
Go to the location where the libvma package was saved.
Run the command below to start installation:
• For RPM packages:
#rpm -i libvma-X.Y.Z-R.<arch>.rpm
• For DEB packages:
#dpkg -i libvma_X.Y.Z-R_<arch>.deb
During the installation process:
• VMA library is copied to standard library location (e.g. /usr/lib64/libvma.so). In addition VMA debug library is copied under the same location (e.g. /usr/lib64/libvma-debug.so)
• README.txt and version information (VMA_VERSION) are installed at /usr/share/doc/libvma-X.Y.Z/• VMA installs its configuration file, libvma.conf, to the following location: /etc/libvma.conf• The vmad service utility is copied into /sbin• The script vma is installed under /etc/init.d/. This script can be used to load and unload the VMA service utility.
Installing the VMA utils Package
Go to the location where the utils package was saved.
Run the command below to start installation:
• For RPM packages:
#rpm -i libvma-utils-X.Y.Z-R.<arch>.rpm
• For DEB packages:
#dpkg -i libvma-utils_X.Y.Z-R_<arch>.deb
During the installation process the VMA monitoring utility is installed at /usr/bin/vma_stats.
Installing the VMA devel Package
Go to the location where the devel package was saved.
Run the command below to start installation:
• For RPM packages:
#rpm -i libvma-devel-X.Y.Z-R.<arch>.rpm
• For DEB packages:
#dpkg -i libvma-dev_X.Y.Z-R_<arch>.deb
During the installation process the VMA extra header file is installed at /usr/include/mellanox/vma_extra.h.
Installing VMA in RHEL 7.x Inbox
VMA can be installed on RHEL 7.2 and above Inbox Drivers.
RHEL 7.x distribution contains RoCE drivers version which can be retrieved by running “yum install”. The NVIDIA drivers contain a code that was tested and integrated into the Linux upstream. However, this code may be more or less advanced than MLNX_OFED version 4.0-x.x.x or MLNX-EN v3.4-x.x.x.
This option is suitable for users who do not wish to change their exiting NVIDIA driver that comes with RHEL 7.x installation.
RHEL 7.2 with Inbox drivers, please see here.
For running VMA over RHEL 7.3 or later with Inbox drivers, execute:
yum install libvma
For libvma configuration file please visit here.
For information on how to uninstall VMA in RHEL 7.x Inbox, please refer to VMA in RHEL 7.x Inbox Uninstallation.
By default “yum install libvma” will install one of past VMA versions. In order to run with latest VMA version, you can compile libvma directly from GitHub.. For further information, refer to Verifying VMA Compilation.
To configure NVIDIA ConnectX adapter card ports to work with the desired transport, please refer to the section Port Type Management/VPI Cards Configuration.