High Throughput Recording

Table of Contents

• Determining the Required the USB Disk Speed
• Options for Capturing High-Throughput Data
• Achieving the Maximum USB Throughput

This section provides guidance on developing strategies for recording the huge amounts of data that multiple cameras can produce.

Determining the Required the USB Disk Speed

Because of storage limitations with the platform's local file system, sensor data is stored on an external USB 3.0 drive. But the drive must be fast enough to keep up with the amount of data that your sensors generate. For example, four cameras generating RAW data at 30 frames per second can exceed the bandwidth of most SSD USB drives.

The required write speed of the drives depends on the number of sensors and the recording format. If you are recording RAW data, calculate the required drive speed as:

width * height * 2 * fps = bytes/second

RAW recording consumes most of the bandwidth.

Options for Capturing High-Throughput Data

The following table provides suggestions for recording more than three cameras generating RAW.

Cameras generating RAW	Framerate	Options
3-4	30	Attach all cameras to a single Xavier and mount a USB on that Xavier.
6-8	30	Attach 3-4 cameras to Xavier A and mount a USB on that Xavier. Do the same for Xavier B. Use PTP or gPTP to synchronize camera timestamps between the Xavier cameras.
12	30	Use two NVIDIA DRIVE platforms. For each platform, follow the steps as for 6-8 cameras at 30 FPS.

Achieving the Maximum USB Throughput

You should be able to achieve USB3.0-SSD write speeds between 400 and 500- megabytes per second, depending on the brand of the USB3.0 SSD being used. That speed is adequate for three cameras generating RAW data. If you need to store more data than your USB device can handle, consider distributing recording across the different Xavier processors. For more information, see Distributed Recording in this guide.

NVIDIA has tested extensively with Samsung 850 EVO and Samsung 750 EVO SSDs, where the SSDs are formatted with the EXT4 file system. The following are guidelines for ensuring optimal disk speed.

The system automatically mounts the USB drive.

To initialize your USB disk (Linux)

1. Enable the CFQ schedule for the USB disk:

   # echo cfq > /sys/block/sda/queue/scheduler
   

   The directory /sys/block/device/queue/iosched contains files that allow the administrator to retrieve and set tunable values related to the I/O scheduler.
2. Change the memory limit for the I/O scheduler.

   # echo 1000 > /sys/module/usbcore/parameters/usbfs_memory_mb
   

3. Confirm you successfully updated the memory limit.

   # cat /sys/module/usbcore/parameters/usbfs_memory_mb
   

To optimize your platform to store recorded data (Linux)

1. Locate and edit the following file:

   /etc/sysctl.conf
   

2. Modify the file to have the following settings. These are persistent changes and are preserved between reboots.

   net.core.rmem_default = 1048576
   net.core.rmem_max = 10485760
   net.core.wmem_default = 1048576
   net.core.wmem_max = 10485760
   net.core.netdev_max_backlog = 30000
   net.ipv4.ipfrag_high_thresh = 8388608
   net.ipv6.conf.all.disable_ipv6 = 1
   vm.dirty_background_ratio = 5
   vm.dirty_ratio = 50

To determine the write speed of the mounted jbod disk (Linux)

• Enter:

   # dd if=/dev/zero of=/media/NVidia/folder#/file bs=1G count=10
  

  The JBod disk is mounted on:

   /media/NVidia/