To set up SR-IOV, do the following:
Download mstflint
# zypper install mstflint
Check the device’s PCI
lspci | grep Mellanox
Example:
24:00.0 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex]
Check if SR-IOV is enabled in the in the firmware.
mstconfig -d <device pci> q
Example:
# mstconfig -d 00:06.0 q Device #1: Device type: ConnectX3Pro PCI device: 00:06.0 Configurations: Current SRIOV_EN True(1) NUM_OF_VFS 8 LINK_TYPE_P1 ETH(2) LINK_TYPE_P2 IB(1) LOG_BAR_SIZE 3 BOOT_PKEY_P1 0 BOOT_PKEY_P2 0 BOOT_OPTION_ROM_EN_P1 True(1) BOOT_VLAN_EN_P1 False(0) BOOT_RETRY_CNT_P1 0 LEGACY_BOOT_PROTOCOL_P1 PXE(1) BOOT_VLAN_P1 1 BOOT_OPTION_ROM_EN_P2 True(1) BOOT_VLAN_EN_P2 False(0) BOOT_RETRY_CNT_P2 0 LEGACY_BOOT_PROTOCOL_P2 PXE(1) BOOT_VLAN_P2 1 IP_VER_P1 IPv4(0) IP_VER_P2 IPv4(0)
Enable SR-IOV:
mstconfig –d <device pci> s SRIOV_EN=<False|True>
Configure the needed number of VFs.
mstconfig –d <device pci> s NUM_OF_VFS=<NUM>
This file will be generated only if IOMMU is set in grub configuration. (by adding “intel_iommu=on” to /etc/default/grub).
[mlx4 devices only] Edit the file /etc/modprobe.d/mlx4.conf:
options mlx4_core num_vfs=[needed num of VFs] port_type_array=[1/2
Example:
options mlx4_core num_vfs=8 port_type_array=1,1
[mlx5 devices only] Write to the sysfs file the number of needed
echo [num_vfs] > /sys/class/infiniband/mlx5_0/device/sriov_numvfs
- Reboot the
Load the driver and verify that the VFs were
lspci | grep mellanox
Example:
24:00.0 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex] 24:00.1 Infiniband controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex] 24:00.2 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function] 24:00.3 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function] 24:00.4 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function] 24:00.5 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
For further information, refer to section "Setting Up SR-IOV" in the MLNX_OFED User Manual.
