Secure Firmware Update

Warning

Secure Firmware Update is supported only on ConnectX-4 onwards adapter cards.

A “Secure firmware update” is the ability of a device to verify digital signatures of new firmware binaries, in order to assure that only officially approved versions can be installed from the host, the network[1] or a Board Management Controller (BMC).

The firmware of devices with “secure firmware up date” functionality (secure FW), restricts access to specific commands and registers that can be used to modify the firmware binary image on the flash, as well as commands that can jeopardize security in general. Most notably, the commands and registers for random flash access are disabled.

Secure FW verifies new binaries before activating them, compared to legacy devices where this task is done by the update tool using direct flash access commands. In addition to signature verification, secure FW also checks that the binary is designated to the same device model, that the new firmware is also secured, and that the new FW version is not included in a forbidden versions blacklist. The firmware rejects binaries that do not match the verification criteria.

Secure FW utilizes the same ‘fail safe’ upgrade procedures, so events like power failure during update should not leave the device in an unstable state. The table below lists the impact of secure FW update on MFT tools.

Tool

Flow

Secure FW

With CS Token

Blocked Commands

flint / mlxburn

Burn FW

Working with controlled fw update

Working with controlled fw update

Query

Working with controlled fw update

Working with controlled fw update

Set GUIDs

Working with controlled fw update

Working with controlled fw update

Verify

Working partially (BOOT image)

Working partially (BOOT image)

Set DV INFO: SET MFG, SET VSD, VPD

Not supported in Secure FW

Not supported in Secure FW

MFBA

ROM OPS: BROM, DROM

Not supported, BOOT image modification is not supported (MFBA)

Not supported, BOOT image modification is not supported (MFBA)

MFBA

"-ocr" override cache replacement (Direct flash GW access)

Not supported in Secure FW

Not supported in Secure FW

Flash GW is blocked

HW SET (Set flash parameters)

Flash GW is blocked

Flash GW is blocked

Flash GW is blocked

"--no_fw_ctrl" (Legacy Flow)

Not supported in Secure FW

Not supported in Secure FW

MFBA

mlxfwmanager / mlxup

Burn FW

Working with controlled fw update

Working with controlled fw update

mlxfwmanager

with --no_fw_ctrl

Not supported in Secure FW

Not supported in Secure FW

MFBA

mlxdump

fsdump

Blocked icmds

Working

gcif_get_ft_info, gcif_get_ft_list, gcif_get_fg, gcif_get_fg_list, gcif_get_fte, gcif_get_fte_list

phyUc

Blocked icmds

working

gcif_phy_uc_get_array_prop_px, gcif_phy_uc_set_get_data,
gcif_phy_uc_get_array_prop_EDR, gcif_phy_uc_get_array_prop_HDR

rxdump

CR-Space is locked & Blocked icmds

working

gcif_read_rx_slice_desc, gcif_read_rx_slice_packet

sxdump

CR-Space is locked & Blocked icmds

working

gcif_read_wq_buf fer

wqdump

Dump QP contexts

Blocked icmds

working

gcif_read_context

Dump WQs

Blocked icmds

working

gcif_read_host_m em, gcif_read_q_en- try, gcif_qp_get_pi_ci

ICM

Blocked icmds

working

gcif_read_icm

WRITE QP (Devmon)

working

gcif_write_context

mget_temp

hw_access

Read Only CR- Space

working

Read Only CR- Space

mcra

Read

working

working

working

Write

Read Only CR- Space

working

Read Only CR- Space

mstdump

Read

working

working

working

mlxtrace / fwtrace

MEM & FIFO

Only fwtrace is supported and only in Linux

working

Read Only CR- Space

pckt_drop

uses write to CR- Space to work

Read Only CR- Space

working

Read Only CR- Space

mlxlink

working

working

working

working

mlxreg

working

working

working

working

mlxcables

working

working

working

working

mlxconfig

working

working

working

working

mlxfwreset

working

working

working

working

i2c/mlxi2c

Not relevant when not in livefish

With Force flag (ENV VAR)

Read Only CR- Space

working

Read Only CR- Space

The following sections describe how Secure FW updates are performed.

For firmware Secure purposes, you may sign the image file using the sign command. If you do not provide the sign command with a private key and UUID, the command will only compute SHA256 digest and add it to the image signature section. The sign command supports RSA keys with lengths of 2048 and 4096 bits.

  • If you provide a private key with the length of 2048 bits, the command will compute SHA256 digest and encrypt it with the private key and add the result with the provided UUID to the appropriate image signature section.

  • If you provide a private key with the length of 4096 bits, the command will compute SHA512 digest and encrypt it with the provided key and add the result with the provided UUID to the appropriate image signature.

You can sign with two keys in the same command by providing keys with lengths of 2048 and 4096 bits. The flags to be used for the first private key and uuid are “--private_key“ and “--key_uuid”, and for the second private and uuid use “--private_key2” and “–key_uuid2”.

The motivation for signing with two keys is to allow a firmware update from both firmwares, the one that supports only 2048bit keys and the one that supports 4096bit keys.

Examples:

Copy
Copied!
            

# flint -i /tmp/image.bin sign --private_key privatekey.pem --key_uuid "e0129552-13ba-11e7-a990-0cc47a6d39d2"

Copy
Copied!
            

# flint -i /tmp/image.bin sign --private_key privatekey_2048.pem --key_uuid "e0129552-13ba-11e7-a990-0cc47a6d39d2" --private_key2 privatekey_4096.pem --key_uuid2 "a0b43568-17cb-16e9-a990-0ff47a6d39e4"

Hardware Security Module (HSM) can be used with flint as well trough OpenSSL dynamic engines to allow the user to sign the firmware with a secure key management system instead of directly providing key files to flint.

Warning

HSM and OpenSSL engines should be configured properly to work with dynamic engines.

Example:

Copy
Copied!
            

# flint -i /tmp/image.bin --openssl_engine pkcs11 --openssl_key_id "pkcs11:serial=0123456789abcdef;token=My%20token%201;type=private;object=example_pkey;id=%12%34%56%78" --key_uuid "e0129552-13ba-11e7-a990-0cc47a6d39d2" sign

To override the public keys section in a given binary image file, use set_public_key.

Copy
Copied!
            

# flint -i /tmp/image.bin set_public_keys public_key.bin

To override the forbidden versions section in a given binary image file, use set_forbidden_versions.

Copy
Copied!
            

# flint -i /tmp/image.bin set_forbidden_versions forbidden_versions.bin

When Secure Firmware is enabled, the flint output slightly changes due to the differences in the underlying NIC accessing methods. Some functionalities may be restricted according to the device security level.
flint query under secure mode:

Copy
Copied!
            

# flint -d /dev/mst/mt4115_pciconf0 q Image type: FS3 FW Version: 12.19.2278 FW Release Date: 7.6.2017 Description: UID GuidsNumber Base GUID: 7cfe90030029205e 4 Base MAC: 00007cfe9029205e 4 Image VSD: Device VSD: PSID: MT_2190110032 Security Attributes: secure-fw, dev

Warning

Unavailable information is reported as N/A.

In secure firmware, a firmware update will be successful if an image is signed with a valid key that is recognized by the running firmware on the chip. for more information, please refer to Signing Binary Image Files.If the security type permits legacy flash access commands, the --no_fw_ctrl flag can be used to command the flint to work in the non firmware controlled mode. This means that all the non-secure functionality will be supported using this flag, and the burn flow will work without requiring a signed image.Example:

Copy
Copied!
            

# flint -d /dev/mst/mt4115_pciconf0 --no_fw_ctrl q Image type: FS3 FW Version: 12.19.2096 FW Release Date: 26.3.2017 Description: UID GuidsNumber Base GUID: 248a07030094050c 4 Base MAC: 0000248a0794050c 4 Image VSD: Device VSD: PSID: MT_2170110021

The following procedure is intended to be implemented by customers who want to use their keys to sign a secured firmware.

  1. Set the public keys in a given firmware image:

    1. Generate a binary file that contains 8 public keys.
      You can use mlxconfig command xml2bin to generate the file:

      1. To generate 2048 bits public keys:

        1. Run: mlxconfig gen_tlvs_file output.txt.

        2. Open the output.txt.

        3. Go to the line starting with "file_public_key" and change the 0 to 1.

        4. Save the file and exit.

        5. Run: mlxconfig gen_xml_template output.txt output.xml

        6. Open the output.xml.

        7. Duplicate the xml node "file_public_key" so the file has 8 copies, for each node fill it as follows:
          • cs_token_en = 0
          • fw_en = 1
          • mlnx_nvconf_en = 1
          • vendor_nvconf_en = 1
          • auth_type: 0x3 for 2048 bits keys and 0x4 for 4096 bits keys.
          Example for public_key_exp, keypair_uuid, key:

          Copy
          Copied!
                      

          <public_key_exp>4083403379</public_key_exp> <keypair_uuid>5A7A2B2A87DB7416</keypair_uuid> <key> f8000003000000000000000000010001c459afea005911e797dc000000000000b8168ba624e5cac81d491f48c6a3b8f1a816cb7dea789d770893b0fb5abeb67f7a8d19ad8d4203dd8b85b3faaaf96187b116eb1c5d3f3517c3ce8b4422395f2e43ccb286d4bc4474c8385e857349f35be3094f25ccbd71c209c6531f0d8bcaacdbbf14af58809e8937e4db424b3d0c48e0cae7b89f53f797b9e24335900448466b0e5182e3a94c31e18487f8fe367862c8a70e8c7007d2400760461bbb36470a26d6db13d2e63d137d67cd449c0788c307ce2dbc3f580ec7207cdb856472520ee956912cfaf77e6e793f620d6e362fa13da036003f85ae8dbb22d4b314ceb64c </key>

          Warning

          You can have spaces between the bytes: f8 00 00 03, or you can have multiple lines.
          The order of the bytes is the same as the output of openssl file, Therefore, you can take the key as is from the openssl file.

        8. Save and Exit.

        9. Run: mlxconfig xml2bin output.xml output.bin.

      2. To generate 4096 bits public keys, please follow the same steps as above, but use "file_public_key_4096" instead of "file_public_key".
        For further information, see mlxconfig xml2bin Command.

    2. Set the key's binary file in the firmware image using the flint set_public_keys command.
      For further information, see Setting a “Public Keys” Section in a Binary Image File.

  2. If there is need to modify the definition for the forbidden_versions in a given firmware image then:

    1. Generate a binary file that contains the forbidden versions.
      You can use the mlxconfig command xml2bin to generate it according to the steps described in Step a above (Generate a binary file that contains 8 public keys).
      An example for forbidden versions xml node:

      Copy
      Copied!
                  

      <nv_forbidden_versions> <creation_time_day>18</creation_time_day> <creation_time_month>6</creation_time_month> <creation_time_year>7e2</creation_time_year> <creation_time_second>d</creation_time_second> <creation_time_minute>19</creation_time_minute> <creation_time_hour>12</creation_time_hour> <min_allowed_fw_version>0</min_allowed_fw_version> <forbidden_fw_version index="0">53:1f:0d06</forbidden_fw_version> <forbidden_fw_version index="1..31">0</forbidden_fw_version> </nv_forbidden_versions>

    2. Set the key's binary file in the firmware image using the flint set_forbidden_versions command.
      For further information, see Setting a "Forbidden Versions" Section in a Binary Image File.

  3. Sign the firmware image with a private key.

    Warning

    Please notice that signing the image must be after setting the public keys, and the forbidden versions. For further information, see securefmupdate.

    1. Run the flint sign command.

      Warning

      To sign with a 2048 bits private key only, make sure that the firmware image does not contain a 4096 bits key signature.
      Run the flint set_public_keys command with a 4096 bits keys section filled with zeros and then sign with a 2048 bits private key.
      For further information, see Signing Binary Image Files.

      Warning

      To sign with 4096 bits private key only, run the flint set_public_keys command with a 2048 bits keys section filled with zeros and then sign with the a 4096 bits private key.
      For further information, see Signing Binary Image Files.

© Copyright 2023, NVIDIA. Last updated on May 23, 2023.