image image image image image

On This Page

The Mellanox mediated devices deliver flexibility in allowing to create accelerated devices without SR-IOV on the Bluefield® system. These mediated devices support NIC and RDMA, and offer the same level of ASAP2 offloads as SR-IOV VFs. Mediates devices are supported using mlx5 sub-function acceleration technology.

Configuring Mediated Device

  1. To support sub-functions, PCIe BAR2 must be enabled. Run: 

    $ mlxconfig -d /dev/mst/mst41682_pciconf0 s PF_BAR2_SIZE 4 PF_BAR2_ENABLE=True

    Cold reboot the BlueField host system so that the above settings can be applied on subsequent reboot sequences.

  2. By default, the firmware allows for a large number of maximum mdev devices. You must set the maximum number of mediated devices to 2 or 4 devices. Run: 

    $ echo 4 > /sys/bus/pci/devices/0000:05:00.0/mdev_supported_types/mlx5_core-local/max_mdevs

  3. Mediated devices are uniquely identified using UUID. To create one, run:

    $ uuidgen
$ echo 49d0e9ac-61b8-4c91-957e-6f6dbc42557d > /sys/bus/pci/devices/0000:05:00.0/mdev_supported_types/mlx5_core-local/create

  4. By default, if the driver vfio_mdev is loaded, newly created mdev devices are bound to it. To make use of this newly created mdev device in order to create a netdevice and RDMA device, you must first unbind it from that driver. Run:

    $ echo 49d0e9ac-61b8-4c91-957e-6f6dbc42557d > /sys/bus/mdev/drivers/vfio_mdev/unbind

  5. Configure a MAC address for the mdev device. Run:

    $ echo 00:11:22:33:44:55 > /sys/bus/mdev/devices/49d0e9ac-61b8-4c91-957e-6f6dbc42557d/devlink-compat-config/mac_addr

  6. Query the representor netdevice of the mdev device. Run:

    $ cat /sys/bus/mdev/devices/49d0e9ac-61b8-4c91-957e-6f6dbc42557d/devlink-compat-config/netdev

  7. Bind the mediated device to mlx5_core driver. Run:

    $ echo 49d0e9ac-61b8-4c91-957e-6f6dbc42557d > /sys/bus/mdev/drivers/mlx5_core/bind

    When an mdev device is bound to the mlx5_core driver, its respective netdevice and/or RDMA device is also created.

  8. To inspect the netdevice and RDMA device for the mdev, run: 

    $ ls /sys/bus/mdev/devices/49d0e9ac-61b8-4c91-957e-6f6dbc42557d/net/
$ ls /sys/bus/mdev/devices/49d0e9ac-61b8-4c91-957e-6f6dbc42557d/infiniband/