Quick Start Guide
NVIDIA® DeepStream Software Development Kit (SDK) is an accelerated AI framework to build intelligent video analytics (IVA) pipelines. DeepStream runs on NVIDIA® T4 and platforms such as NVIDIA® Jetson™ Nano, NVIDIA® Jetson AGX Xavier™, NVIDIA® Jetson Xavier NX™, NVIDIA® Jetson™ TX1 and TX2.
Jetson Setup
This section explains how to prepare a Jetson device before installing the DeepStream SDK.
To install Jetson SDK components
• Download NVIDIA SDK Manager, which you will use to install JetPack 4.4 GA (corresponding to L4T 32.4.3 release) from:
https://developer.nvidia.com/embedded/jetpack
NVIDIA SDK Manager is a graphical application which flashes and installs the JetPack packages.
The flashing procedure takes approximately 10-30 minutes, depending on the host system.
To install additional packages
• Enter the following commands to install the prerequisite packages:
$ sudo apt install \
libssl1.0.0 \
libgstreamer1.0-0 \
gstreamer1.0-tools \
gstreamer1.0-plugins-good \
gstreamer1.0-plugins-bad \
gstreamer1.0-plugins-ugly \
gstreamer1.0-libav \
libgstrtspserver-1.0-0 \
libjansson4=2.11-1
To install librdkafka (to enable Kafka protocol adaptor for message broker)
1. Clone the librdkafka repository from GitHub:
$ git clone https://github.com/edenhill/librdkafka.git
2. Configure and build the library:
$ cd librdkafka
$ git reset --hard 7101c2310341ab3f4675fc565f64f0967e135a6a
./configure
$ make
$ sudo make install
3. Copy the generated libraries to the deepstream directory:
$ sudo mkdir -p /opt/nvidia/deepstream/deepstream-5.0/lib
$ sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream-5.0/lib
To install latest NVIDIA V4L2 GStreamer plugin
4. Open the apt source configuration file in a text editor, for example:
$ sudo vi /etc/apt/sources.list.d/nvidia-l4t-apt-source.list
5. Change the repository name and download URL in the deb commands shown below:
deb https://repo.download.nvidia.com/jetson/common r32.4 main
deb https://repo.download.nvidia.com/jetson/<platform> r32.4 main
Where <platform> identifies the platform’s processor:
• t186 for Jetson TX2 series
• t194 for Jetson AGX Xavier series or Jetson Xavier NX
• t210 for Jetson Nano or Jetson TX1
For example, if your platform is Jetson Xavier NX:
• deb https://repo.download.nvidia.com/jetson/common r32.4 main
• deb https://repo.download.nvidia.com/jetson/t194 r32.4 main
6. Save and close the source configuration file.
7. Enter the commands:
$ sudo apt update
$ sudo apt install --reinstall nvidia-l4t-gstreamer
If apt prompts you to choose a configuration file, reply Y for yes (to use the NVIDIA updated version of the file).
Note: | Updating NVIDIA V4L2 GStreamer plugin should be performed after flashing Jetson OS from SDK Manager. |
To install the DeepStream SDK
• Method 1: Using SDK Manager
Select DeepStreamSDK from the “Additional SDKs” section along with JP 4.4 software components for installation.
• Method 2: Using the DeepStream tar package
1. Download the DeepStream 5.0 Jetson tar package deepstream_sdk_v5.0.0_jetson.tbz2, to the Jetson device.
2. Enter the following commands to extract and install DeepStream SDK:
$ sudo tar -xvf deepstream_sdk_v5.0.0_jetson.tbz2 -C /
$ cd /opt/nvidia/deepstream/deepstream-5.0
$ sudo ./install.sh
$ sudo ldconfig
• Method 3: Using the DeepStream Debian package
Download the DeepStream 5.0 Jetson Debian package deepstream-5.0_5.0.0-1_arm64.deb, to the Jetson device. Then enter the command:
$ sudo apt-get install ./deepstream-5.0_5.0.0-1_arm64.deb
Note: | If you install the DeepStream SDK Debian package using the dpkg command, you must install the following packages before installing the DeepStream deb package: • libgstrtspserver-1.0-0 • libgstreamer-plugins-base1.0-dev |
To boost the clocks
After you have installed DeepStream SDK, run these commands on the Jetson device to boost the clocks:
$ sudo nvpmodel -m 0
$ sudo jetson_clocks
To run deepstream-app (the reference application)
1. Navigate to the samples directory on the development kit
2. Enter the following command to run the reference application:
$ deepstream-app -c <path_to_config_file>
Where <path_to_config_file> is the pathname of one of the reference application’s configuration files, found in configs/deepstream-app/. See Package Contents for a list of the available files.
Note: | • You can find sample configuration files under /opt/nvidia/deepstream/deepstream-5.0/samples directory. Enter this command to see application usage: $ deepstream-app --help |
| • To save TensorRT Engine/Plan file, run the following command: $ sudo deepstream-app -c <path_to_config_file> |
• To show labels in 2D Tiled display view, expand the source of interest with mouse left-click on the source. To return to the tiled display, right-click anywhere in the window.
• Keyboard selection of source is also supported. On the console where application is running, press the ‘z’ key followed by the desired row index (0 to 9), then the column index (0 to 9) to expand the source. To restore 2D Tiled display view, press ‘z’ again.
To run precompiled sample applications
1. Navigate to the to the chosen application directory inside sources/apps/sample_apps
2. Follow the directory’s README file to run the application.
Note: | If the application encounters errors and cannot create Gst elements, remove the GStreamer cache, then try again. To remove the GStreamer cache, enter this command: $ rm ${HOME}/.cache/gstreamer-1.0/registry.aarch64.bin When the application is run for a model which does not have an existing engine file, it may take up to a few minutes depending on the platform and the model for the engine file to be generated and application to start playing. For later runs, these generated engine files can be reused for faster loading. |
dGPU Setup for Ubuntu
This section explains how to prepare an Ubuntu x86_64 system with NVIDIA dGPU devices before installing the DeepStream SDK.
Note: | This document uses the term dGPU (“discrete GPU”) to refer to NVIDIA GPU expansion card products such as NVIDIA® Tesla® T4 and P4, NVIDIA® GeForce® GTX 1080, and NVIDIA® GeForce® RTX 2080. This version of DeepStream SDK runs on specific dGPU products on x86_64 platforms supported by NVIDIA driver 450.51 and NVIDIA® TensorRT™ 7.0 and later versions. |
You must install the following components:
• Ubuntu 18.04
• GStreamer 1.14.1
• NVIDIA driver 450.51
• CUDA 10.2
• TensorRT 7.0.X
To remove all previous DeepStream installations
Enter the following commands to remove all previous DeepStream 3.0 or prior installations:
$ sudo rm -rf /usr/local/deepstream /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstnv* /usr/bin/deepstream* /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libnvdsgst*
/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream*
/opt/nvidia/deepstream/deepstream*
$ sudo rm -rf /usr/lib/x86_64-linux-gnu/libv41/plugins/libcuvidv4l2_plugin.so
To remove DeepStream 4.0 or later installations:
1. Open the uninstall.sh file in /opt/nvidia/deepstream/deepstream/
2. Set PREV_DS_VER as 4.0
3. Run the following script as sudo: ./uninstall.sh
To install packages
• Enter the following commands to install the necessary packages before installing the DeepStream SDK:
$ sudo apt install \
libssl1.0.0 \
libgstreamer1.0-0 \
gstreamer1.0-tools \
gstreamer1.0-plugins-good \
gstreamer1.0-plugins-bad \
gstreamer1.0-plugins-ugly \
gstreamer1.0-libav \
libgstrtspserver-1.0-0 \
libjansson4
To install NVIDIA driver 450.51
• Download and install NVIDIA driver 450.51 from NVIDIA unix drivers page at:
• Run the following commands:
$chmod 755 NVIDIA-Linux-x86_64-450.51.run
$./ NVIDIA-Linux-x86_64-450.51.run
After installation you must set environment variable INDIVIDUAL_DECODER_SCHEDULING to TRUE (for T4 only).
To install CUDA Toolkit 10.2
• Download and install CUDA Toolkit 10.2 from NVIDIA Developer Center at:
To install TensorRT 7.0
• Download and install TensorRT 7.0 from the NVIDIA Developer Center:
To install librdkafka (to enable Kafka protocol adaptor for message broker)
1. Clone the librdkafka repository from GitHub:
$ git clone https://github.com/edenhill/librdkafka.git
2. Configure and build the library:
$ cd librdkafka
$ git reset --hard 7101c2310341ab3f4675fc565f64f0967e135a6a
./configure
$ make
$ sudo make install
3. Copy the generated libraries to the deepstream directory:
$ sudo mkdir -p /opt/nvidia/deepstream/deepstream-5.0/lib
$ sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream-5.0/lib
To install the DeepStream SDK
• Method 1: Using the DeepStream Debian package
Download the DeepStream 5.0 dGPU Debian package, deepstream-5.0_5.0.0-1_amd64.deb, then enter the command:
$ sudo apt-get install ./deepstream-5.0_5.0.0-1_amd64.deb
• Method 2: Using the DeepStream tar package
Navigate to the location of the downloaded DeepStream package to extract and install the DeepStream SDK:
$ sudo tar -xvf deepstream_sdk_v5.0.0_x86_64.tbz2 -C /
$ cd /opt/nvidia/deepstream/deepstream-5.0/
$ sudo ./install.sh
$ sudo ldconfig
To run the deepstream-app (the reference application)
• Go to the samples directory and enter this command:
$ deepstream-app -c <path_to_config_file>
Where <path_to_config_file> is the pathname of one of the reference application’s configuration files, found in configs/deepstream-app. See Package Contents for a list of the available files.
Note: | • To dump engine file, run the following command: $ sudo deepstream-app -c <path_to_config_file> • You can find sample configuration files under /opt/nvidia/deepstream/deepstream-5.0/samples directory. Enter this command to see application usage: $ deepstream-app --help |
• To show labels in 2D tiled display view, expand the source of interest with a mouse left-click on the source. To return to the tiled display, right-click anywhere in the window.
• Keyboard selection of source is also supported. On the console where application is running, press the ‘z’ key followed by the desired row index (0 to 9), then the column index (0 to 9) to expand the source. To restore the 2D Tiled display view, press ‘z’ again.
To run precompiled sample applications
1. Navigate to the chosen application directory inside sources/apps/sample_apps.
2. Follow that directory’s README file to run the application.
Note: | If the application encounters errors and cannot create Gst elements, remove the GStreamer cache, then try again. To remove the GStreamer cache, enter this command: $ rm ${HOME}/.cache/gstreamer-1.0/registry.x86_64.bin When the application is run for a model which does not have an existing engine file, it may take up to a few minutes depending on the platform and the model for the engine file to be generated and application to start playing. For later runs, these generated engine files can be reused for faster loading. |
dGPU Setup for RedHat Enterprise Linux
This section explains how to prepare an RHEL system with NVIDIA dGPU devices before installing the DeepStream SDK.
Note: | This document uses the term dGPU (“discrete GPU”) to refer to NVIDIA GPU expansion card products such as NVIDIA Tesla T4 and P4, NVIDIA GeForce GTX 1080, and NVIDIA GeForce RTX 2080. This version of DeepStream SDK runs on specific dGPU products on x86_64 platforms supported by NVIDIA driver 450+ and NVIDIA® TensorRT™ 7.0 and later versions. |
You must install the following components:
• RHEL 8.x
• GStreamer 1.14.1
• NVIDIA driver 450.51 (https://www.nvidia.com/en-us/drivers/unix/linux-amd64-display-archive/)
• CUDA 10.2
• TensorRT 7.0.X
To remove all previous DeepStream installations
To remove DeepStream 4.0 or later installations:
1. Open the uninstall.sh file which will be present in /opt/nvidia/deepstream/deepstream/
2. Set PREV_DS_VER as 4.0
3. Run the following script as sudo: ./uninstall.sh
To install packages
• Enter the following command to install the necessary packages before installing the DeepStream SDK:
$ yum install \
gstreamer1 \
gstreamer1-plugins-base \
gstreamer1-plugins-good \
gstreamer1-plugins-bad-free \
gstreamer1-plugins-ugly-free \
gstreamer1-rtsp-server \
gstreamer1-svt-av1 \
json-glib \
openssl \
libuuid \
gstreamer1-plugins-base-devel \
json-glib-devel \
opencv-devel \
jansson-devel \
openssl-devel \
libuuid-devel
gst-rtsp-server:
---------------
gst-rtsp-server-devel package is not available for RHEL which is required to compile deepstream-app
Download sources from https://gstreamer.freedesktop.org/src/gst-rtsp-server/gst-rtsp-server-1.14.5.tar.xz
$ ./configure
$ make
$ make install
$ sudo cp -r /usr/local/include/gstreamer-1.0/gst/rtsp-server/ /usr/include/gstreamer-1.0/gst/
Note: | Installation of packages required for RHEL 8.x are also mentioned in README.rhel in DeepStream package. |
To install NVIDIA driver 450.51
• Download and install NVIDIA driver 450.51 from NVIDIA unix drivers page at:
• Run the following commands:
$chmod 755 NVIDIA-Linux-x86_64-450.51.run
$./ NVIDIA-Linux-x86_64-450.51.run
After installation you must set the environment variable INDIVIDUAL_DECODER_SCHEDULING to TRUE (for T4 only).
To install CUDA Toolkit 10.2
• Download and install CUDA Toolkit 10.2 from NVIDIA Developer Center at:
To install TensorRT 7.0
• Download and install TensorRT 7.0 from the NVIDIA Developer Center:
To install librdkafka (to enable Kafka protocol adaptor for message broker)
1. Clone the librdkafka repository from GitHub:
$ git clone https://github.com/edenhill/librdkafka.git
2. Configure and build the library:
$ cd librdkafka
$ git reset --hard 7101c2310341ab3f4675fc565f64f0967e135a6a
./configure
$ make
$ sudo make install
3. Copy the generated libraries to the deepstream directory:
$ sudo mkdir -p /opt/nvidia/deepstream/deepstream-5.0/lib
$ sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream-5.0/lib
To install the DeepStream SDK
Using the DeepStream tar package:
1. Navigate to the location to which the DeepStream package was downloaded and extract and install the DeepStream SDK:
$ sudo tar -xvf deepstream_sdk_v5.0.0_x86_64.tbz2 -C /
$ cd /opt/nvidia/deepstream/deepstream-5.0/
$ sudo ./install.sh
$ sudo ldconfig
To run the deepstream-app (the reference application)
• Go to the samples directory and enter this command:
$ deepstream-app -c <path_to_config_file>
Where <path_to_config_file> is the pathname of one of the reference application’s configuration files, found in configs/deepstream-app. See Package Contents for a list of the available files.
Note: | • To dump engine file run the following command: $ sudo deepstream-app -c <path_to_config_file> • You can find sample configuration files under /opt/nvidia/deepstream/deepstream-5.0/samples directory. Enter this command to see application usage: $ deepstream-app --help |
• To show labels in 2D tiled display view, expand the source of interest with a mouse left-click on the source. To return to the tiled display, right-click anywhere in the window.
• Keyboard selection of source is also supported. On the console where application is running, press the ‘z’ key followed by the desired row index (0 to 9), then the column index (0 to 9) to expand the source. To restore the 2D Tiled display view, press ‘z’ again.
To run precompiled sample applications
1. Navigate to the chosen application directory inside sources/apps/sample_apps.
2. Follow that directory’s README file to run the application.
Note: | If the application encounters errors and cannot create Gst elements, remove the GStreamer cache, then try again. To remove the GStreamer cache, enter this command: $ rm ${HOME}/.cache/gstreamer-1.0/registry.x86_64.bin When the application is run for a model which does not have an existing engine file, it may take up to a few minutes depending on the platform and the model for the engine file to be generated and application to start playing. For later runs, these generated engine files can be reused for faster loading. |
Running without an X server
The default configuration files provided with the SDK have the EGL based nveglglessink as the default renderer (indicated by type=2 in the [sink] groups). The renderer requires a running X server and fails without one.
In case of absence of an X server, DeepStream reference applications provide an alternate functionality of streaming the output over RTSP. This can be enabled by adding an RTSP out sink group in the configuration file. Refer to [sink2] group in source30_1080p_dec_infer-resnet_tiled_display_int8.txt file for an example. Don’t forget to disable the nveglglessink renderer by setting enable=0 for the corresponding sink group.
Platform and OS Compatibility
The following table provides information about platform and operating system compatibility in the current and earlier versions of DeepStream.
NVIDIA Jetson Platforms |
DeepStream release | DeepStream 1.0 | DeepStream 1.5 | DeepStream 2.0 | DeepStream 3.0 | DeepStream 4.0.1 (Unified) | DeepStream 4.0.2 (Unified) | DeepStream 5.0 GA (Unified) |
Jetson
platforms | TX2, TX1 | TX2, TX1 | Not supported | AGX Xavier | Nano, AGX Xavier, TX2, TX1 | Nano, AGX Xavier, TX2, TX1 | Nano, AGX Xavier, TX2, TX1, Jetson NX |
OS | L4T Ubuntu 16.04 | L4T Ubuntu 16.04 | Not supported | L4T Ubuntu 18.04/16.04 | L4T Ubuntu 18.04 | L4T Ubuntu 18.04 | L4T Ubuntu 18.04 |
JetPack release | 3.1 | 3.2 | Not supported | 4.1.1 | 4.2.1 | 4.3 | 4.4 GA |
L4T release | 28.1 | 28.2 | Not supported | 31.1 | 32.2 | 32.3.1 | 32.4.3 |
CUDA release | CUDA 8.0 | CUDA 9.0 | Not supported | CUDA 10.0 | CUDA 10.0 | CUDA 10.0 | CUDA 10.2 |
cuDNN release | cuDNN 6.0 | cuDNN 7.0.5 | Not supported | cuDNN 7.3 | cuDNN 7.5.1 | cuDNN 7.6.3 | cuDNN 8.0.0.x |
TensorRT release | TRT 2.1 | TRT 3.0 | Not supported | TRT 5.0 | TRT 5.1.6 | TRT 6.0.1 | TRT 7.1.3 |
OpenCV release | OpenCV 2.4.13 | OpenCV 3.3.1 | Not supported | OpenCV 3.3.1 | OpenCV 3.3.1 | OpenCV 4.1 | OpenCV 4.1.1 |
VisionWorks | VisionWorks 1.6 | VisionWorks 1.6 | Not supported | VisionWorks 1.6 | VisionWorks 1.6 | VisionWorks 1.6 | VisionWorks 1.6 |
GStreamer | GStreamer 1.8.3 | GStreamer 1.8.3 | Not supported | GStreamer 1.8.3 | GStreamer 1.14.1 | GStreamer 1.14.1 | GStreamer 1.14.1 |
Docker image | Not available | Not available | Not available | Not available | deepstream-l4t:4.0 | deepstream-l4t:4.0.2 | deepstream-l4t:5.0 |
dGPU Platforms |
|---|
DeepStream release | DeepStream 1.0 | DeepStream 1.5 | DeepStream 2.0 | DeepStream 3.0 | DeepStream 4.0.1 (Unified) | DeepStream 4.0.2 (Unified) | DeepStream 5.0 GA (Unified) |
|---|
GPU platforms | P4, P40 | P4, P40 | P4, P40 | P4, P40, V100, T4 | P4, T4, V100 | P4, T4, V100 | P4, T4, V100 |
OS | Ubuntu 16.04 | Ubuntu 16.04 | Ubuntu 16.04 | Ubuntu 16.04 | Ubuntu 18.04 | Ubuntu 18.04 | Ubuntu 18.04 RHEL 8.x |
GCC | GCC 5.4 | GCC 5.4 | GCC 5.4 | GCC 5.4 | GCC 7.3.0 | GCC 7.3.0 | GCC 7.3.0 |
CUDA release | CUDA 8.0 | CUDA 9.0 | CUDA 9.2 | CUDA 10.0 | CUDA 10.1 | CUDA 10.1 | CUDA 10.2 |
cuDNN release | cuDNN 6.0 | cuDNN 7.0 | cuDNN 7.1 | cuDNN 7.3 | cuDNN 7.5.0+ | cuDNN 7.6.5+ | cuDNN 7.6.5+ |
TRT release | TRT 2.1 | TRT 3.0 | TRT 4.0 | TRT 5.0 | TRT 5.1.5 | TRT 6.0.1 | TRT 7.0.0 |
Display Driver | R375 | R384 | R396+ | R410+ | R418+ | R418+ | R450.51 |
VideoSDK release | SDK 7.1 | SDK 7.9 | SDK 7.9 | SDK 8.2 | SDK 9.0 | SDK 9.0 | SDK 9.1 |
OFSDK release | Not available | Not available | Not available | Not available | 1.0.10 | 1.0.10 | 1.0.10 |
GStreamer release | Not available | GStreamer 1.8.3 | GStreamer 1.8.3 | GStreamer 1.8.3 | GStreamer 1.14.1 | GStreamer 1.14.1 | GStreamer 1.14.1 |
OpenCV release | Not available | OpenCV 2.4.13 | OpenCV 3.4.x | OpenCV 3.4.x | OpenCV 3.3.1 | OpenCV 3.3.1 | OpenCV 3.4.0 |
Docker image | Not available | Not available | Not available | deepstream:3.0 | deepstream:4.0 | deepstream:4.0.2 | deepstream:5.0 |
DeepStream Triton Inference Server Usage Guidelines
dGPU:
1. Pull the DeepStream Triton Inference Server docker
docker pull nvcr.io/nvidia/deepstream:5.0.0-20.07-triton
2. Start the docker
docker run --gpus all -it --rm -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY nvcr.io/nvidia/deepstream:5.0.0-20.07-triton
Jetson:
The Triton Inference Server shared libraries come pre-installed as part of DeepStream on Jetson. No extra steps are required for installing the Triton Inference Server.
For both the platforms, to run the samples follow the steps in the “Running the Triton Inference Server samples” section of the README at: /opt/nvidia/deepstream/deepstream-5.0