Storage
The storage service enables auto setup and provisioning of external storage (such as hard disks) attached to a Jetson device, and provided as a systemd service called jetson-storage. Before enabling or starting the jetson-storage service you may want to customize few configurations in Customizations first. And the service detects and formats available storages at the next boot automatically after you enable jetson-storage service by:
sudo systemctl enable jetson-storage
Then it configures the storage device according to the config file /opt/nvidia/jetson-configs/jetson-storage.conf and /opt/nvidia/jetson-configs/storage-quota.json at subsequent boots until you disable the service by running:
sudo systemctl disable jetson-storage
Note
The storage drive will be mounted under /data. If there is an existing /data directory in the root file system, its contents will get hidden by the mounted storage. It is recommended to delete any existing content in /data before starting storage service, as that will continue to occupy disk space in the root file system.
The jetson-storage service will create a symlink /var/lib/docker to /data/docker-volume. If you already have docker running before starting jetson-storage service, the docker service will be stopped, and all the containers will be terminated. You may start the docker service manually by “systemctl start docker” or reboot your device, so the service can be up and running again. The existing files in original /var/lib/docker directory will be moved to /data/docker-volume by jetson-storage service, so the docker images you downloaded previously will be kept.
Note
When removing package nvidia-jetson-services the symlink /var/lib/docker will be removed. A new /var/lib/docker directory will be created when restarting docker service, so you won’t be able to access any previous data in /data/docker-volume.
Factory Reset
If you want to use a drive that was used earlier by storage service and want to erase the old data and use it like a new drive, reset the disk label (see following examples) and restart the storage service, so it will format the drive and re-setup the filesystem for storage.
With LVM disabled
SATA: sudo e2label /dev/sda1 ""
NVMe: sudo e2label /dev/nvme0n1p1 ""
With LVM enabled
Enter command sudo pvdisplay to check if your storage had been formatted as a LVM volume for JPS. If you see output as below, and the VG Name is jetson-vg, you will need to remove the LVM first:
--- Physical volume ---
PV Name /dev/sda1
VG Name jetson-vg
PV Size <14.96 GiB / not usable 2.00 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 3829
Free PE 0
Allocated PE 3829
PV UUID J5Dgo3-205q-xLA9-z5QF-C7On-bHSK-mA9c1
Enter the command sudo vgremove -y jetson-vg to remove VG jetson-vg. Then remove the PV by:
SATA: sudo pvremove -y /dev/sda1
NVMe: sudo pvremove -y /dev/nvme0n1p1
Replace /dev/sda1 and /dev/nvme0n1p1 in the example above with your storage path. If you have multiple storages, you will need to perform the operation for each one of them.
Customizations
jetson-storage.conf
There are few options which can be changed in the /opt/nvidia/jetson-configs/jetson-storage.conf:
LVM_ENABLED=1
ENCRYPTION_ENABLED=0
# User defined storage list, ex: ("/dev/sda" "/dev/nvme0n1" "dev/sdb")
# Only feasible when ENCRYPTION_ENABLED=0
STORAGE_PATH=()
MAX_STORAGE_NUM=5
The LVM_ENABLED is used to enable LVM for multiple disks. This option is set by default since this release. If you’re upgraded from prior release with LVM disabled, you need to edit /opt/nvidia/jetson-configs/jetson-storage.conf and set it as 1 manually. Note that if the disk partition was formatted as a LVM volume before please follow the instructions in Factory Reset to remove the LVM parition first, otherwise the jetson-storage service will get confused by unknown LVM volume and behave improperly.
If the LVM_ENABLED is set you may add more disks into your system after setting up jetson storage for the first time as long as the total storage number is smaller than (or equal to) the MAX_STORAGE_NUM as mentioned below. The new storages will be formatted properly and added into LVM. We don’t suggest you to remove or replace any LVM devices unless you have good experience in advanced LVM operations. Later when you have more disks to add into your system after jetson-storage started, you may reboot your device so the service can setup your new disks at boot. Or you may restart the service to add new disks by:
sudo systemctl restart jetson-storage
The ENCRYPTION_ENABLED is used to enable Disk Encryption on Jetson storages. Please refer to Disk Encryption.
The STORAGE_PATH is an array in Bash. The option is only feasible when Disk Encryption is disabled (ENCRYPTION_ENABLED is 0). You can list available disks on your system by command sudo blkid and specify the storage you want to use before enabling jetson-storage , e.g.:
STORAGE_PATH=("/dev/sda")
If the STORAGE_PATH is empty (as the default), the jetson-storage service will automatically pick available storages as Jetson Storage.
The MAX_STORAGE_NUM is the maximal storage number you would like to support in jetson-storage service. Please update the value if you have more storage drives connected to your Jetson than MAX_STORAGE_NUM. The numer of storage drives in STORAGE_PATH can’t exceed MAX_STORAGE_NUM.
storage-quota.json
The default /opt/nvidia/jetson-configs/storage-quota.json includes items for few microservices, and this file will be overwritten by application bundle.
Below is one example from ai_nvr to fill in the required fields in storage-quota.json. Anyone can add their own setting in the bottom of the file. The service will create a system user with name and user id, and in subsequent release will configure disk quota for the user with specified quota in KiB. The NVME or SATA HDD partition is mounted to /data, and within that are sub-directories which are named by volume for various Jetson software modules. Below is the default configuration:
{
"default-group" : "logusers",
"default-group-id" : "150",
"jetson-quota" : [
{
"name" : "ingress",
"id" : 2002,
"quota" : 1048576,
"volume" : "ingress-secrets-volume",
"in-group" : "logusers"
},
{
"name" : "redis",
"id" : 2003,
"quota" : 5242880,
"volume" : "redis-volume",
"in-group" : "logusers"
},
{
"name" : "logging",
"id" : 2004,
"quota" : 10485760,
"volume" : "logging-volume",
"in-group" : "logusers",
"share-volume" : "true"
},
{
"name" : "monitoring",
"id" : 2005,
"quota" : 5242880,
"volume" : "monitoring-volume",
"dirs" : "db",
"in-group" : "logusers"
},
{
"name": "vodi",
"id" : 2007,
"quota" : 104857600,
"volume" : "vodi-volume",
"in-group" : "logusers"
},
{
"name": "iotgateway",
"id" : 2008,
"quota" : 1048576,
"volume" : "iotgateway-volume",
"in-group" : "logusers"
}
]
}
Note
The defined storage quotas are not enforced in the first release.
Disk Encryption
You can follow the instructions below to enable Disk Encryption on Jetson Storage.
Flashing
Please refer to section Security > Disk Encryption in Jetson Linux Developer Guide and flash your Jetson device with Disk Encryption first. Below is an example to enable Disk Encryption on Jetson AGX Orin and Jetson Orin Nx:
Generate EKS image according to Jetson Linux Developer Guide and overwrite the
Linux_for_Tegra/bootloader/eks.imgandLinux_for_Tegra/bootloader/eks_t234.img.Generate default EKB key:
cd Linux_for_Tegra echo "00000000000000000000000000000000" > ekb.key
Note
The zero key above is just an example. You should replace it with whatever you set when generating EKS image.
Put device into recovery modules
Flash Jetson AGX Orin:
sudo ROOTFS_ENC=1 ./flash.sh -i "./ekb.key" jetson-agx-orin-devkit mmcblk0p1
Flash Jetson Orin Nx:
sudo ./tools/kernel_flash/l4t_initrd_flash.sh --network usb0 --showlogs -p "-c bootloader/generic/cfg/flash_t234_qspi.xml" -i ./ekb.key --no-flash jetson-orin-nano-devkit internal sudo ROOTFS_ENC=1 ./tools/kernel_flash/l4t_initrd_flash.sh --network usb0 --showlogs --no-flash --external-device nvme0n1p1 -S 16GiB -c ./tools/kernel_flash/flash_l4t_t234_nvme_rootfs_enc.xml --external-only --append -i ./ekb.key jetson-orin-nano-devkit external sudo ./tools/kernel_flash/l4t_initrd_flash.sh --network usb0 --showlogs --flash-only
Note
By default root file system will be excrypted. You can enable Disk Encryption only for external storages and leave root file system parition as unencrypted. Please refer to Jetson Linux Developer Guide for more details.
Enablement
Install nvidia-jetson-services package as described in Quick Start Guide.
Edit /opt/nvidia/jetson-configs/jetson-storage.conf and set ENCRYPTION_ENABLED as 1:
ENCRYPTION_ENABLED=1
Then enable jetson-storage service:
sudo systemctl enable jetson-storage
Power-off your Jetson device, make sure the storage is inserted properly, and then power-on the device. Note that the jetson-storage would need to detect and add your storage into LUKS table at first boot after enabling the jetson-storage, so you will need to check the service status by:
systemctl status jetson-storage
If the status is active (exited) as below, reboot your Jetson again so that Jetson Disk Encryption utility could format and encrypt your device properly.
ubuntu@tegra-ubuntu:~$ systemctl status jetson-storage
● jetson-storage.service - Jetson storage service
Loaded: loaded (/lib/systemd/system/jetson-storage.service; enabled; vendor preset: enabled)
Active: active (exited) since Thu 2023-11-23 02:49:37 UTC; 4 days ago
Process: 4315 ExecStart=/opt/nvidia/jetson/services/storage/bin/jetson-storage-setup init (code=exited, status=0/SUCCESS)
Main PID: 4315 (code=exited, status=0/SUCCESS)
CPU: 597ms
All available external storages will be detected and setup as Jetson Storage by jetson-storage service.
Now when you login the console you should see the mounted message like below. You’re ready to use your encrypted storage now.
CURRENT STORAGE STATUS ON JETSON:
* Mount Point: /data
* Mount Path: /dev/mapper/jetson-enc-nvme0n1
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/jetson-enc-nvme0n1 1.9T 48K 1.8T 1% /data