Access to Hardware Devices
The table below lists the NVIDIA® devices supported by MFT, the supporting tools, and the access methods to these devices.
Device Type |
Product Name |
HW Access Method |
||
PCI |
I2C |
In-Band |
||
HCA (InfiniBand) |
NVIDIA Connect-IB |
V |
V |
V |
IB/ETH Network Adapter |
NVIDIA ConnectX®-3 |
V |
V |
V |
NVIDIA ConnectX-3 Pro |
V |
V |
V |
|
NVIDIA ConnectX-4 |
V |
V |
V |
|
NVIDIA ConnectX-5 |
V |
V |
V |
|
NVIDIA ConnectX-5 Ex |
V |
V |
V |
|
NVIDIA ConnectX-6 |
V |
V |
V |
|
Ethernet Adapter (NIC) |
NVIDIA ConnectX-3 EN |
V |
V |
|
NVIDIA ConnectX-4 Lx |
V |
V |
||
NVIDIA ConnectX-6 Dx |
V |
V |
||
NVIDIA ConnectX-6 Lx |
V |
V |
||
Switch |
NVIDIA SwitchX®-2 |
V2 |
V |
V |
NVIDIA Switch-IB® |
V1 |
V |
V |
|
NVIDIA Switch-IB 2 |
V1 |
V |
V |
|
NVIDIA Spectrum™ |
V |
V |
||
NVIDIA Spectrum-2 |
V |
V |
||
NVIDIA Spectrum-3 |
V |
V |
||
NVIDIA Quantum |
V |
V |
V |
Note. V1 indicates managed switch products only.MFT tools access NVIDIA devices via the PCI Express interface, via a USB to I2C adapter (P/N: MTUSB-1), or via vendor-specific MADs over the InfiniBand fabric (In-Band).
In-Band device access requires the local IB port to be in the ACTIVE state and connected to an IB fabric.
All MFT tools address the target hardware device using an mst device name. This name is assigned by running the command ‘mst start’ (in Windows, it is not required to run the “mst start” command) for PCI and I2C access. In-Band devices can be assigned by running the 'mst ib add' command. In-Band devices can be assigned by running the 'mst ib add' command.
To list the available mst device names on the local machine, run ‘mst status’. Local PCI devices may also be accessed using device aliases. Supported aliases are:
PCI device “bus:dev.fn” (e.g. 03:00.0)
OFED RDMA device (e.g. mlx4_0)
Network interface with “net-” prefix (e.g. net-eth2)
Run mst status -v to list the devices and their available aliases. The format of an mst device name is as follows:
Via PCI:
# mt4099_pci_crX
# mt4099_pciconf0
where:
X is the index of the adapter on the machine.
_crX devices access the adapter directly (recommended if possible)
_pciconfX devices use configuration cycles to access the adapter
For example:
# mt25418_pci_cr0
Via USB to I2C adapter: For example, mtusb-1.
Via Remote device:
/dev/mst/mft:23108
,@dev
@mst
@mt4103_pci_cr0
Via ibdr device: For example, /dev/mst/CA_MT4113_server1_HCA-3_ibdr-0,mlx- 5_0,1 or ibdr-0,mlx5_0,1.
Via In-Band:
For example:
/dev/mst/CA_MT4099_mft_HCA-1_lid-0x0002
or simply "lid-2"
"
The “mst ib add” command adds devices in the format:
For adapters:
CA_<device id >_<ib node description>_lid-<lid number>
For switches:
SW_<device id >_lid-<lid number>
See Step 3 in Remote Access to NVIDIA Devices for instructions on how to obtain the device LID.
Via PCI user level:
For example, if you run lspci -d 15b3: NVIDIA devices and PCI Device IDs will be displayed.
# /sbin/lspci -d 15b3:
02
:00.0
Ethernet controller: Mellanox Technologies Unknown device 6368
(rev a0)