Access to Hardware Devices

The table below lists the NVIDIA® devices supported by MFT, the supporting tools, and the access methods to these devices.

Device Type

Product Name

HW Access Method

PCI

I2C

In-Band

HCA (InfiniBand)

NVIDIA Connect-IB

V

V

V

IB/ETH Network Adapter

NVIDIA ConnectX®-3

V

V

V

NVIDIA ConnectX-3 Pro

V

V

V

NVIDIA ConnectX-4

V

V

V

NVIDIA ConnectX-5

V

V

V

NVIDIA ConnectX-5 Ex

V

V

V

NVIDIA ConnectX-6

V

V

V

Ethernet Adapter (NIC)

NVIDIA ConnectX-3 EN

V

V

NVIDIA ConnectX-4 Lx

V

V

NVIDIA ConnectX-6 Dx

V

V

NVIDIA ConnectX-6 Lx

V

V

Switch

NVIDIA SwitchX®-2

V2

V

V

NVIDIA Switch-IB®

V1

V

V

NVIDIA Switch-IB 2

V1

V

V

NVIDIA Spectrum™

V

V

NVIDIA Spectrum-2

V

V

NVIDIA Spectrum-3

V

V

NVIDIA Quantum

V

V

V

Note. V1 indicates managed switch products only.MFT tools access NVIDIA devices via the PCI Express interface, via a USB to I2C adapter (P/N: MTUSB-1), or via vendor-specific MADs over the InfiniBand fabric (In-Band).

Warning

In-Band device access requires the local IB port to be in the ACTIVE state and connected to an IB fabric.

All MFT tools address the target hardware device using an mst device name. This name is assigned by running the command ‘mst start’ (in Windows, it is not required to run the “mst start” command) for PCI and I2C access. In-Band devices can be assigned by running the 'mst ib add' command. In-Band devices can be assigned by running the 'mst ib add' command.

To list the available mst device names on the local machine, run ‘mst status’. Local PCI devices may also be accessed using device aliases. Supported aliases are:

  • PCI device “bus:dev.fn” (e.g. 03:00.0)

  • OFED RDMA device (e.g. mlx4_0)

  • Network interface with “net-” prefix (e.g. net-eth2)

Run mst status -v to list the devices and their available aliases. The format of an mst device name is as follows:

Via PCI:

Copy
Copied!
            

# mt4099_pci_crX

Copy
Copied!
            

# mt4099_pciconf0

where:

X is the index of the adapter on the machine.
_crX devices access the adapter directly (recommended if possible)
_pciconfX devices use configuration cycles to access the adapter

For example:

Copy
Copied!
            

# mt25418_pci_cr0

Via USB to I2C adapter: For example, mtusb-1.

Via Remote device:

Copy
Copied!
            

/dev/mst/mft:23108,@dev@mst@mt4103_pci_cr0 

Via ibdr device: For example, /dev/mst/CA_MT4113_server1_HCA-3_ibdr-0,mlx- 5_0,1 or ibdr-0,mlx5_0,1.

Via In-Band: lid-.

For example:

Copy
Copied!
            

/dev/mst/CA_MT4099_mft_HCA-1_lid-0x0002 or simply "lid-2""

The “mst ib add” command adds devices in the format:

  • For adapters:

    Copy
    Copied!
                

    CA_<device id >_<ib node description>_lid-<lid number>

  • For switches:

    Copy
    Copied!
                

    SW_<device id >_lid-<lid number>

See Step 3 in Remote Access to NVIDIA Devices for instructions on how to obtain the device LID.

Via PCI user level:

For example, if you run lspci -d 15b3: NVIDIA devices and PCI Device IDs will be displayed.

Copy
Copied!
            

# /sbin/lspci -d 15b3: 02:00.0 Ethernet controller: Mellanox Technologies Unknown device 6368 (rev a0)

© Copyright 2023, NVIDIA. Last updated on Oct 12, 2023.