Secure Host

Linux Kernel Upstream Release Notes v5.17

Secure host (supported in ConnectX®-3/ConnectX®-3 Pro adapter cards only) enables the device to protect itself and the subnet from malicious software. This is achieved by:

  • Not allowing untrusted entities to access the device configuration registers, directly (through pci_cr or pci_conf) and indirectly (through MADs)

  • Hiding the M_Key from the untrusted entities

  • Preventing the modification of GUID0 by the untrusted entities

  • Preventing drivers on untrusted hosts to receive or transmit SMP (QP0) MAD packets (SMP firewall)

When the SMP firewall is enabled, the firmware handles all QP0 packets, and does not forward them to the driver. Any information required by the driver for proper operation (e.g., SM lid) is passed via events generated by the firmware while processing QP0 MADs.
Driver support mainly requires using the MAD_DEMUX firmware command at driver startup.

Secure mode capability is enabled by setting the "cr_protection_en" parameter set to 1 in the [HCA] section of the .ini file and then burning the firmware with this .ini file. If the parameter is set to zero, or is missing, secure-mode operation will not be possible.
Once the firmware allows secure-mode operation, the secure-mode capability must be activated by using "flint" to set a 64-bit key (and then restarting the driver).
The flint command is as follows (the key is specified as up to 16 hex digits):

Copy
Copied!
            

flint -d <device> set_key <64-bit key>

Example:

Copy
Copied!
            

flint -d /dev/mst/mt26428_pci_cr0 set_key 1a1a1a1a2b2b2b2b

Enabling/Disabling Hardware Access

Once a 64-bit key is installed, the secure-mode is active once the driver is restarted. If the host is rebooted, the HCA comes out of reboot with secure-mode enabled. Hardware access can be disabled while the driver is running to enable operation such as maintenance, or firmware burning and then restored at the end of the operation.

Warning

The temporary enable does not affect the SMP firewall. This remains active even if the "cr-space" is temporarily permitted.

Procedure_Heading_Icon.PNG

To enable hardware access:

Copy
Copied!
            

flint -d /dev/mst/mt26428_pci_cr0 hw_access enable Enter Key: ********

disable hardware access:

Copy
Copied!
            

flint -d /dev/mst/mt26428_pci_cr0 hw_access disable

Warning

If you do not explicitly restore hardware access when the maintenance operation is completed, the driver restart will NOT do so. The driver will come back after restart with hardware access disabled. Note, though, that the SMP firewall will remain active.

A host reboot will restore hardware access (with SMP firewall active). Thus, when you disable hardware access, you should restore it immediately after maintenance has been completed, either by using the flint command above or by rebooting the host (or both).

Burning New Firmware when Secure-Mode is Active

  1. Temporarily enable the hardware access (see Enabling/Disabling Hardware Access section).

  2. Burn the new firmware.

  3. Reboot the host (not just restart the driver).

Permanently Disabling Secure-Mode

  1. Temporarily disable secure-mode (see Enabling/Disabling Hardware Access section).

  2. Reset the pass-key to zero.

    Copy
    Copied!
                

    flint -d <device> set_key 0 

  3. Reboot the host.

This operation will cause the HCA to always come up (even from host reboot) in insecure mode. To restore security, simply set a non-zero pass-key again.

Checking if Hardware Access is Active

:

Copy
Copied!
            

flint -d /dev/mst/mt26428_pci_cr0 -qq q

If the hardware access is active, you will see the following error message:

Copy
Copied!
            

E- Cannot open /dev/mst/mt26428_pci_cr0: HW access is disabled on the device. E- Run "flint -d /dev/mst/mt26428_pci_cr0 hw_access enable" in order to enable HW access.

Checking if the SMP Firewall is Active

The SMP firewall is active as long as there is a non-zero pass-key active in the firmware (regardless of whether or not the Secure-Mode has been temporarily disabled).
To check if SMP Firewall is active, run the InfiniBand diagnostic command sminfo.
If the SMP firewall is active, the command will fail as shown below:

Copy
Copied!
            

[root@dev-l-vrt-016 ~]# sminfo ibwarn: [26872] mad_rpc: _do_madrpc failed; dport (Lid 1) sminfo: iberror: failed: query

© Copyright 2023, NVIDIA. Last updated on Oct 11, 2023.