Managing Self-Encrypting Drives

The NVIDIA DGX OS software supports the ability to manage self-encrypting drives (SEDs), including setting an Authentication Key for locking and unlocking the drives on NVIDIA DGX H100, DGX A100, DGX A800, DGX Station A100, and DGX-2 systems.

You can manage only the SED data drives. The software cannot be used to manage OS drives even if they are SED-capable.

Overview

The SED management software is in the nv-disk-encrypt package.

The software supports the following configurations:

  • NVIDIA DGX H100, DGX A100, DGX A800, DGX Station A100, and DGX-2 systems where all data drives are self-encrypting drives.

  • Only SEDs used as data drives are supported.

    The software will not manage SEDs that are OS drives.

The software provides the following functionality:

  • Identifies eligible drives on the system.

  • Allows you to you assign Authentication Keys (passwords) for each SED as part of the initialization process.

    • Alternatively, the software can generate random passwords for each drive.

    • The passwords are stored in a password-protected vault on the system.

  • Once initialized, SEDs are locked upon power loss, such as a system shutdown or drive removal.

    Locked drives get unlocked after power is restored and the root file system is mounted.

  • Provides functionality to export the vault.

  • Provides functionality for erasing the drives.

  • Provides the ability to revert the initialization.

Installing the Software

Use the package manager to install the nv-disk-encrypt package and, optionally, the TPM2 tools package, and reboot the system. You need the TPM tools package if you plan to use the TPM2 to store security keys.

  1. Update the packages.

    sudo apt update

  2. Install nv-disk-encrypt.

    sudo apt install -y nv-disk-encrypt

  3. (Optional) Install the TPM tools package.

    • For DGX A100, DGX Station A100, or DGX H100, install the tpm2-tools package.

    sudo apt install -y tpm2-tools

    • For DGX-2, install the tpm-tools package.

    sudo apt install -y tpm-tools

  4. Reboot.

    sudo reboot

If you plan to use TPM2, enable it. Refer to Configuring Trusted Computing for more information.

Configuring Trusted Computing

Here is some information about the controls that are required to configure Trusted Computing (TC).

The DGX H100 system BIOS provides setup controls for configuring the following TC features:

  • Trusted Platform Module

    The NVIDIA DGX H100, DGX A100, DGX A800, and DGX Station A100 incorporate Trusted Platform Module 2.0 (TPM 2.0). The DGX-2 incorporates a TPM module. These modules can be enabled from the system BIOS and used in conjunction with the nv-disk-encrypt tool. After being enabled, the nv-disk-encrypt tool uses the TPM for encryption and stores the vault and SED authentication keys on the TPM instead of on the file system. Using the TPM is preferred because this allows the vault data to persist even if the system is reimaged.

  • Block SID

    Certain drives shipped with the DGX systems support the Block SID authentication feature. Block SID authentication prevents malicious actors from taking ownership of drives and blocks others from using the drives. By default, the DGX BIOS will send the Block SID request. On such setups, you will need to enable the Disable Block Sid feature in the BIOS before proceeding with the initialization steps.

Note

Enabling the “Disable Block SID” option is only valid for one reboot, so if drive encryption needs to be enabled again, then the feature needs to be enabled in BIOS as well.

Determining Whether Drives Support SID

The drive model is a good indicator of whether the drive supports this feature. Issue the following and look for one of the following model strings:

  • KCM6DRUL3T84

  • KCM6DRUL7T68

  • MZQLB7T6HMLA-00007

sudo nvme list

Node          SN                            Model
------------- ----------------------------- -----
/dev/nvme0n1  70H0A0AHTTHR KCM6DRUL3T84 ...
/dev/nvme1n1  70H0A007TTHR KCM6DRUL3T84

Enabling the TPM and Preventing the BIOS from Sending Block SID Requests

This section provides instructions to enable the TPM and prevent the SBIOS from sending Block SID request. Each task is independent, so you can select which task to complete.

  1. Reboot the system, then press [Del] or [F2] at the NVIDIA splash screen to enter the BIOS Setup.

  2. Navigate to the Advanced tab on the top menu, then scroll to Trusted Computing and press [Enter].

    _images/trusted-computing-1.png

    • To enable TPM, scroll to Security Device and switch the setting to Enabled.

    _images/trusted-computing-2.png

    • To disable Block SID, go to the Security tab to the right, and scroll to Disable Block Sid, then switch to Enabled.

    _images/trusted-computing-3.png

  3. Save and exit the BIOS Setup to continue the boot process.

    If you disabled Block SID, you will be prompted to accept the request to disable issuing a Block SID Authentication command.

    _images/access-sbios-setup.png

  4. Press F10 at the prompt. After the system boots, you can proceed to initialize drive encryption.

Initializing the System for Drive Encryption

Here is some information about how to initialize the system for drive encryption.

Note

Before initializing drive encryption, review the information in Configuring Trusted Computing and follow the configuration instructions if needed.

Initialize the system for drive encryption using the nv-disk-encrypt command.

sudo nv-disk-encrypt init [-k <your-vault-password>] [-f <path/to/json-file>] [-g] [- r]

Here is a list of the options:

  • -k: Lets you create the vault password in the command.

    Otherwise, the software will prompt you to create a password before proceeding.

  • -f: Lets you specify a JSON file that contains a mapping of passwords to drives.

    Refer to “Example 1: Passing in the JSON File” for further instructions.

  • -g: Generates random salt values (stored in /etc/nv-disk-encrypt/.dgxenc.salt) for each drive password.

    Salt values are characters added to a password for enhanced password security. NVIDIA strongly recommends using this option for best security, otherwise the software will use a default salt value instead of a randomly generated one.

  • -r: Generates random passwords for each drive.

    This avoids the need to create a JSON file or the need to enter a password one by one during the initialization.

Enabling Drive Locking

After initializing the system for SED management, issue the following command, which uses the nv-disk-encrypt command to enable drive locking.

sudo nv-disk-encrypt lock

After initializing the system and enabling drive locking, the drives will become locked when they lose power. The system will automatically unlock each drive when power is restored to the system and the system is rebooted.

Initialization Examples

This section provides some initialization examples.

Example 1: Passing in the JSON File

The following instructions in this section describe a method to specify the drive/password mapping ahead of time. This method is useful for initializing several drives at a time and avoids the need to enter the password for each drive after issuing the initialization command, or if you want control of the passwords.

Refer to the following for more information:

Determining Which Drives Can be Managed as Self-Encrypting

Here is some information about how you can determine which drives can be managed as self-encrypting.

Review the storage layout of the DGX system to determine which drives are eligible to be managed as SEDs.

sudo nv-disk-encrypt info

The default output shows which drives can be used for encryption and which drives cannot. The following status information is provided:

  • SED capable: Is this a self-encrypting drive?

  • Boot disk: Is this drive currently being used as a boot drive? Does it contain the root filesystem?

  • Locked: Is this drive currently in the locked state? Is it able to accept I/O?. It can only be in this state after the following conditions have been met:

    • Locking has been enabled (nv-disk-encrypt init, followed by nv-disk-encrypt                                                 init lock)

    • The drive is coming back from power-off.

    • The user queries this state prior to it being (automatically) unlocked.

  • Lock Enabled: Are locks enabled on this drive? It will be in this state after initialization (nv- disk-encrypt init).

  • MBR done: This setting is only relevant for drives that support MBR shadowing. On drives that support this feature, this will report ‘Y’ after initialization (nv-disk-encrypt                                           init)

MBR done: This setting is only relevant for drives that support MBR shadowing. On drives that support this feature, this will report ‘Y’ after initialization (nv-disk-encrypt                                     init)

The following example output snippet shows drives than can be used for encryption. Notice SED capable = Y and Boot disk = N.

_images/show-drives-encryption-use.jpg

The following example output snippet shows drives than cannot be used for encryption. Notice SED capable = Y and Boot disk = Y, or SED capable = N.

_images/show-drives-no-encrypt-use.jpg

Alternatively, you can specify the output be presented in JSON format by using the -j option.

sudo nv-disk-encrypt info -j

In this case, drives that can be used for encryption are indicate by the following:

"sed_capable": true "used_for_boot": false

And drives that cannot be used for encryption are indicated by one of the following:

"sed_capable": true "used_for_boot": true

Or

"sed_capable": false

Creating the Drive/Password Mapping JSON Files and Using it to Initialize the System

You can initialize the system by creating the drive and password map the JSON files.

  1. Create a JSON file that lists all the eligible SED-capable drives that you want to manage.

    Note

    These are the list of drives that you obtained from Determining Which Drives Can be Managed as Self-Encrypting

    The following example shows the format of the JSON file.

    {
"/dev/nvme2n1": "<your-password>",
"/dev/nvme3n1": "<your-password>",
"/dev/nvme4n1": "<your-password>",
"/dev/nvme5n1": "<your-password>",
}

    • Ensure that you follow the syntax exactly.

    • Passwords must consist of only upper-case letters, lower-case letters, digits, and/or the following special characters: ~ : @ % ^ + = _ ,

  2. Initialize the system and then enable locking.

    The following command assumes you have placed the JSON file in the /tmp directory.

    sudo nv-disk-encrypt init -f /tmp/<your-file>.json -g
sudo nv-disk-encrypt lock

    When prompted, enter a password for the vault.

    Passwords must consist of only upper-case letters, lower-case letters, digits, and/or the following special characters: ~ : @ % ^ + = _ ,

Example 2: Generating Random Passwords

The commands in this topic use the -k and -r options so that you are not prompted to enter passwords. You pass the vault password into the command and then the command instructs the tool to generate random passwords for each drive.

The vault password must consist of only upper-case letters, lower-case letters, digits, and/or the following special-characters: ~ : @ % ^ + = _ ,

sudo nv-disk-encrypt init -k <your-vault-password> -g -r
sudo nv-disk-encrypt lock

Example 3: Specifying Passwords One at a Time When Prompted

If there are a small number of drives, or you do not want to create a JSON file, issue the following command.

sudo nv-disk-encrypt init -g
sudo nv-disk-encrypt lock

The software prompts you to enter a password for the vault and then a password for each eligible SED.

Passwords must consist of only upper-case letters, lower-case letters, digits, and/or the following special characters: ~ : @ % ^ + = _ ,

Disabling Drive Locking

To disable drive locking at any time after you initialize, run the following command: $ sudo nv-disk-encrypt disable

This command disables locking on all drives. You can run the initial set up again at any time after this process is complete.

Enabling Drive Locking

After initializing the system for SED management, issue the following command, which uses the nv-disk-encrypt command to enable drive locking.

sudo nv-disk-encrypt lock

After initializing the system and enabling drive locking, the drives will become locked when they lose power. The system will automatically unlock each drive when power is restored to the system and the system is rebooted.

Exporting the Vault

Here is some information about how to export the vault.

To export all drive keys out to a file, use the export function. This requires that you pass in the vault password.

sudo nv-disk-encrypt export -k yourvaultpassword
Writing vault data to /tmp/secrets.out

Erasing Your Data

Here is some information about how you can erase your data.

Warning

Be aware when executing this that all data will be lost. On DGX H100 systems, these drives generally form a RAID 0 array, and this array will also be destroyed when you perform an erase.

After initializing the system for SED management, use the nv-disk-encrypt command to erase data on your drives after stopping cachefilesd and unmounting the RAID array as follows.

  1. Completely stop the RAID.

    systemctl stop cachefilesd
sudo umount /raid
sudo mdadm --stop /dev/md1

  2. Perform the erase.

    sudo nv-disk-encrypt erase

This command does the following:

  • Sets the drives in an unlocked state.

  • Disables locking on the drives.

  • Removes the RAID 0 array configuration.

To rebuild the RAID array, issue the following command:

sudo /usr/bin/configure_raid_array.py -c -f

Clearing the TPM

If you’ve lost the password to your TPM, you will not be able to access its contents. In this case, the only way to regain access to the TPM is to clear the TPM’s contents. After clearing the TPM, you will need to re-initialize the vault and SED authentication keys.

  1. Reboot the system, then press [Del] or [F2] at the NVIDIA splash screen to enter the BIOS Setup.

  2. Navigate to the Advanced tab on the top menu, scroll to Trusted Computing, and press [Enter].

  3. Clear TPM2.

    1. Scroll to Trusted Computing and press [Enter].

    2. Scroll to Pending Operation and press [Enter].

    3. Select TPM Clear at the Pending Operation popup and press [Enter].

  4. Save and exit the BIOS Setup.

Changing Disk Passwords, Adding Disks, or Replacing Disks

The same steps are needed for changing or rotating passwords, adding disks, or replacing disks.

  1. Disable SED management.

    sudo nv-disk-encrypt disable

  2. Add or replace drives as needed and then rebuild the RAID array. Refer to your system’s Service Manual for more information.

  3. Enable SED management and assign passwords per the instructions in Initializing the System for Drive Encryption.

Recovering From Lost Keys

NVIDIA recommends backing up your keys and storing them in a secure location. If you lost the key used to initialize and lock your drives, you will not be able to unlock the drive again. If this happens, the only way to recover is to perform a factory-reset, which will result in a loss of data.

SED drives come with a PSID printed on the label; this value can only be obtained by physically examining the drive as exemplified in the following image.

_images/recover-lost-keys.png

Specify the PSID to reset the drive using the following sedutil-cli command:

sudo sedutil-cli ----yesIreallywanttoERASEALLmydatausingthePSID yourdrivesPSID /dev/nvme3n1