Appendix – Firmware Configuration

NVIDIA BlueField-2 SNAP for NVMe and Virtio-blk v3.8.0

Before configuring mlnx_snap, the user must ensure all FW configuration requirements are met. By default, mlnx_snap is disabled, and needs to be enabled by running both common mlnx-snap configuration, and additional protocol-specific configuration depending on the expected usage of the application (e.g. Hotplug, SR-IOV, UEFI boot, etc).

After all configuration is finished, power-cycling the host is required for these changes to take effect.

Note

To verify that all configuration requirements are satisfied, users may query the current/next configuration by running the following:

Copy
Copied!
            

mlxconfig -d /dev/mst/mt41686_pciconf0 -e query

  1. (Optional) Reset all previous configuration.

    Copy
    Copied!
                

    [dpu] mst start [dpu] mlxconfig -d /dev/mst/mt41686_pciconf0 reset

    Note

    This will return your product to its default configurations. Do this only if you were not able to get SNAP to work.

  2. Set general basic parameters.

    On BlueField-2:

    Copy
    Copied!
                

    [dpu] mlxconfig -d /dev/mst/mt41686_pciconf0 s INTERNAL_CPU_MODEL=1

    On BlueField:

    Copy
    Copied!
                

    [dpu] sudo mlxconfig -d /dev/mst/mt41682_pciconf0 s INTERNAL_CPU_MODEL=1 PF_BAR2_ENABLE=1 PF_BAR2_SIZE=1

  3. When using RDMA/RoCE transport, additional parameters must be configured:

    Copy
    Copied!
                

    [dpu] mlxconfig -d /dev/mst/mt41686_pciconf0 s PER_PF_NUM_SF=1  [dpu] mlxconfig -d /dev/mst/mt41686_pciconf0 s PF_SF_BAR_SIZE=8 PF_TOTAL_SF=2  [dpu] mlxconfig -d /dev/mst/mt41686_pciconf0.1 s PF_SF_BAR_SIZE=8 PF_TOTAL_SF=2 

Parameter

Description

Possible Values

Comments

SRIOV_EN

Enable SR-IOV

0/1

NUM_OF_VFS

Number of VFs per emulated PF

[0-127]

NUM_PF_MSIX

Number of MSIX assigned to emulated PF

[0-63]

NUM_VF_MSIX

Number of MSIX assigned to emulated VF

[0-63]

PCI_SWITCH_EMULATION_ENABLE

Enable PCI switch for emulated PFs

0/1

PCI_SWITCH_EMULATION_NUM_PORT

Max number of emulated PFs

[0-32]

Single port is reserved for all static PFs

Note

SRIOV_EN is valid only for static PFs

Parameter

Description

Possible Values

Comments

NVME_EMULATION_ENABLE

Enable NVMe device emulation

0/1

NVME_EMULATION_NUM_PF

Number of static emulated nvme PFs

[0-2]

NVME_EMULATION_NUM_MSIX

Number of MSIX assigned to emulated NVMe PF/VF

[0-63]

NVME_EMULATION_NUM_VF

Number of VFs per emulated NVMe PF

[0-127]

If not 0, overrides NUM_OF_VFS; valid only when SRIOV_EN=1

EXP_ROM_NVME_UEFI_x86_ENABLE

Enable NVMe UEFI exprom driver

0/1

Used for UEFI boot process

Warning

Due to virtio-blk protocol limitations, using bad configuration while working with static virtio-blk PFs may cause the host server OS to fail on boot.

Before continuing, make sure you have configured:

  • A working channel to access Arm even when the host is shut down. Setting such channel is out of the scope of this document. Please refer to "NVIDIA BlueField DPU Family Software Documentation" for more details.

  • Add the following line to /etc/mlnx_snap/snap_rpc_init.conf:

    Copy
    Copied!
                

    controller_virtio_blk_create mlx5_0 --pf_id 0 --bdev_type none

    For more information, please refer to section “VirtIO-blk Controller Management”.

Parameter

Description

Possible Values

Comments

VIRTIO_BLK_EMULATION_ENABLE

Enable virtio-blk device emulation

0/1

VIRTIO_BLK_EMULATION_NUM_PF

Number of static emulated virtio-blk PFs

[0-2]

See WARNING above

VIRTIO_BLK_EMULATION_NUM_MSIX

Number of MSIX assigned to emulated virtio-blk PF/VF

[0-63]

VIRTIO_BLK_EMULATION_NUM_VF

Number of VFs per emulated virtio-blk PF

[0-127]

If not 0, overrides NUM_OF_VFS; valid only when SRIOV_EN=1

EXP_ROM_VIRTIO_BLK_UEFI_x86_ENABLE

Enable virtio-blk UEFI exprom driver

0/1

Used for UEFI boot process

© Copyright 2024, NVIDIA. Last updated on May 21, 2024.