Mediated Devices

NVIDIA mediated devices deliver flexibility in allowing to create accelerated devices without SR-IOV on the BlueField® system. These mediated devices support NIC and RDMA and offer the same level of ASAP2 offloads as SR-IOV VFs. Mediates devices are supported using mlx5 sub-function acceleration technology.

Two sub-function devices are created on the BlueField device upon boot (one per port if the port is in switchdev mode) using commands from "/etc/mellanox/mlnx-sf.conf" :

Copy
Copied!

            
            /sbin/mlnx-sf -a create -d 0000:03:00.0 -u 61a59715-aeec-42d5-be83-f8f42ba8b049 --mac 12:11:11:11:11:11
/sbin/mlnx-sf -a create -d 0000:03:00.1 -u 5b198182-1901-4c29-97a0-6623f3d02065 --mac 12:11:11:11:11:12

The help menu for mlnx-sf is presented below:

Copy
Copied!

            
            Usage: mlnx-sf [ OPTIONS ]
OPTIONS:
-a, -action,    --action <action>               Perform action
    action:     { enable | create | configure | remove | show | set_max_mdevs | query_mdevs_num }
 
-d, -device,    --device <device>               PCI device <domain>:<bus>:<device>.<func> (E.g.: 0000:03:00.0)
-m, -max_mdevs, --max_mdevs <max mdevs number>  Set maximum number of MDEVs
-u, -uuid,      --uuid <uuid>                   UUID to create SF with
-M, -mac,       --mac <MAC>                     MAC to create SF with
-p, -permanent, --permanent [<conf file>]       Store configuration to be used after reboot and/or driver restart. Default (/etc/mellanox/mlnx-sf.conf).
-V, -version,   --version                       Display script version and exit
-D, -dryrun,    --dryrun                        Display commands only
-v, -verbose,   --verbose                       Run script in verbose mode (print out every step of execution)
-h, -help,      --help                          Display help
 
"/etc/mellanox/mlnx-sf.conf" can be updated manually or using "mlnx-sf" tool with "-p" parameter.

Related Configuration

Interface names are configured using the UDEV rule under: /etc/udev/rules.d/82-net-setup-link.rules.

Copy
Copied!

            
            SUBSYSTEM=="net", ACTION=="add", ATTR{phys_switch_id}!="", ATTR{phys_port_name}!="", \
        IMPORT{program}="/etc/infiniband/vf-net-link-name.sh $attr{phys_switch_id} $attr{phys_port_name}" \
        NAME="$env{NAME}", RUN+="/sbin/ethtool -L $env{NAME} combined 4"
# MDEV network interfaces
ACTION=="add", SUBSYSTEM=="net", DEVPATH=="/devices/pci0000:00/0000:00:00.0/0000:01:00.0/0000:02:02.0/0000:03:00.0/61a59715-aeec-42d5-be83-f8f42ba8b049/net/eth[0-9]", NAME="p0m0"
ACTION=="add", SUBSYSTEM=="net", DEVPATH=="/devices/pci0000:00/0000:00:00.0/0000:01:00.0/0000:02:02.0/0000:03:00.1/5b198182-1901-4c29-97a0-6623f3d02065/net/eth[0-9]", NAME="p1m0"

NVMe SNAP uses p0m0 as its default interface. See /etc/nvme_snap/sf1.conf.