mlx_fpga – Burning and Debugging Tool for NVIDIA Devices with FPGA

NVIDIA ConnectX-5 Adapter Cards Firmware Release Notes v16.35.3502 LTS
Warning

The mlx_fpga utility will be deprecated as of MFT v4.18.0.

mlx_fpga tool allows the user to burn and update a new FPGA image on NVIDIA Innova ™ adapter cards. For instructions on how to burn, please refer to Burning the FPGA’s Flash Device using the mlx_fpga Burning Tool.

The tool also enables the user to read/write individual registers in the FPGA configuration space.

  • An Innova IPsec 4 Lx EN / Innova Flex 4 Lx EN adapter card with an FPGA device

  • For Innova IPsec 4 Lx EN: Load the mlx5_fpga_tools module

  • For Innova Flex 4 Lx EN: Load the mlx_accel_tools module

Note: The module is included in the Innova driver which is supplied for this product line only, and available through the Support.

  • Start mst service with the fpga lookup flag (mst start --with_fpga)

where:

-d|--device <device>

FPGA mst device interface

-v|--version

Display version info

-h|--help

Display help message

-f|--force

Non-interactive mode, answer yes to all questions

r |read <addr>

Read debug register in address

w |write <addr> <data>

Write data to debug register in address

b |burn <image file/s>

Burn the image on the flash

l |load

Load image from flash (--factory - load image from factory flash)

clear_semaphore

Unlock flash controller semaphore

reset

Reset flash controller (--gw) or FPGA (--fpga)

q |query

Query general FPGA information

set_fw_mgmt <disable|enable>

Disable/Enable FPGA management by the Firmware

Adding FPGA mst Device Interface

For Innova IPsec 4 Lx EN: Load the mlx5_fpga_tools module.

Copy
Copied!
            

# modprobe mlx5_fpga_tools # mst start --with_fpga # mst status MST modules: ------------ MST PCI module is not loaded MST PCI configuration module is not loaded MST devices: ------------ No MST devices were found nor MST modules were loaded. You may need to run 'mst start' to load MST modules. FPGA devices: ------------------- /dev/mst/mt4117_pciconf0_fpga_i2c /dev/mst/mt4117_pciconf1_fpga_rdma1

Note: In the last line, it is recommended to use the RDMA device as it is faster. I2C is used for recovery purposes when RDMA is not functional.

For Innova Flex 4 Lx EN: Load the mlx_accel_tools module.

Copy
Copied!
            

# modprobe mlx_accel_tools # mst start --with_fpga # mst status MST modules: ------------ MST PCI module is not loaded MST PCI configuration module is not loaded MST devices: ------------ No MST devices were found nor MST modules were loaded. You may need to run 'mst start' to load MST modules. FPGA devices: ------------------- /dev/mst/mt4117_pciconf0_fpga_i2c /dev/mst/mt4117_pciconf1_fpga_rdma

Note: In the last line, it is recommended to use the RDMA device as it is faster. I2C is used for recovery purposes when RDMA is not functional.

For recovery only, where modules are broken or missing.

Copy
Copied!
            

# mst start --with_fpga_fw_access Starting MST (Mellanox Software Tools) driver set Loading MST PCI module - Success Loading MST PCI configuration module - Success Create devices Unloading MST PCI module (unused) - Success # mst status MST modules: ------------ MST PCI module is not loaded MST PCI configuration module loaded MST devices: ------------ /dev/mst/mt4117_pciconf0 - PCI configuration cycles access. domain:bus:dev.fn=0000:01:00.0 addr.reg=88 data.reg=92 Chip revision is: 00 FPGA devices: ------------------- /dev/mst/mt4117_pciconf0_fpga

Warning

The mlx5_fpga_tools and the mlx_accel_tools modules must be down before trying to recover the FPGA mst Device Interface.


Burning the FPGA’s Flash Device using the mlx_fpga Burning Tool

mlx_fpga tool burns a .bin file onto the FPGA flash device.

The Innova Flex .bin file must be generated according to the instructions listed in the Innova Flex Adapter Card User Manual.

Warning

It is recommended to burn the FPGA device using an RDMA device as it faster and it shortens the burning time.

  1. Burn the image.

    Copy
    Copied!
                

    # mlx_fpga -d <device> burn image.bin

  2. Load the FPGA image from flash according to “Loading the Tool” below or power cycle the machine for change to take effect.

Loading the Tool

Load an FPGA image from user configurable flash:

Copy
Copied!
            

# mlx_fpga -d <device> l/load <optional: load options>

where <optional: load options> is:

--factory

Load FPGA image from factory flash

--user

Load FPGA image from user flash [Default option]


Debugging the Tool

Reading One Debug Register:

Copy
Copied!
            

# mlx_fpga -d <device> read 0x0

Writing One Debug Register:

Copy
Copied!
            

# mlx_fpga -d <device> write 0x0 0x0


Updating the FPGA Image

In order to verify the new image burned to the FPGA, the user can use mlx_fpga tool and read the following registers:

Name

Address

Range

Default

RW

Description

image_version

0x900000

31:00:00

0x0

RO

Version of the image increased on every synthesis.

image_date

0x900004

31:00:00

0x0

RO

Image date of creation.

The hex number is actually the decimal value, i.e. 0x12011995 means 12/01/ 1995 in DD/MM/YY:

bits [31:24] = day of creation bits [23:16] = month of creation bits [15:0] = year of creation

image_time

0x900008

31:00:00

0x0

RO

Image time of creation.

The hex number is actually the decimal value, i.e. 0x00015324 means 01:53:24 in HH:MM:SS:

bits [23:16] = hour (00..23)

bits [15:8] = minutes (00..59)

bits [7:0] = seconds (00..59)

Warning

Innova Flex users should refer to the Innova User Manual for more information.


© Copyright 2023, NVIDIA. Last updated on Dec 28, 2023.