Quickstart Guide¶
NVIDIA® DeepStream Software Development Kit (SDK) is an accelerated AI framework to build intelligent video analytics (IVA) pipelines. DeepStream runs on NVIDIA® T4, NVIDIA® Ampere and platforms such as NVIDIA® Jetson™ Nano, NVIDIA® Jetson AGX Xavier™, NVIDIA® Jetson Xavier NX™, NVIDIA® Jetson™ TX1 and TX2.
Jetson Setup¶
This section explains how to prepare a Jetson device before installing the DeepStream SDK.
Install Jetson SDK components¶
Download NVIDIA SDK Manager from https://developer.nvidia.com/embedded/jetpack. You will use this to install JetPack 4.5.1 GA (corresponding to L4T 32.5.1 release)
NVIDIA SDK Manager is a graphical application which flashes and installs the JetPack packages.
The flashing procedure takes approximately 10-30 minutes, depending on the host system.
Note
If you are using Jetson Nano or Jetson Xavier NX developer kit, you can download the SD card image from https://developer.nvidia.com/embedded/jetpack. This comes packaged with CUDA, TensorRT and cuDNN.
Install Dependencies¶
Enter the following commands to install the prerequisite packages:
$ sudo apt install \
libssl1.0.0 \
libgstreamer1.0-0 \
gstreamer1.0-tools \
gstreamer1.0-plugins-good \
gstreamer1.0-plugins-bad \
gstreamer1.0-plugins-ugly \
gstreamer1.0-libav \
libgstrtspserver-1.0-0 \
libjansson4=2.11-1
Install librdkafka (to enable Kafka protocol adaptor for message broker)¶
Clone the librdkafka repository from GitHub:
$ git clone https://github.com/edenhill/librdkafka.git
Configure and build the library:
$ cd librdkafka $ git reset --hard 7101c2310341ab3f4675fc565f64f0967e135a6a ./configure $ make $ sudo make install
Copy the generated libraries to the deepstream directory:
$ sudo mkdir -p /opt/nvidia/deepstream/deepstream-5.1/lib $ sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream-5.1/lib
Install the DeepStream SDK¶
Method 1: Using SDK Manager
Select DeepStreamSDK from the
Additional SDKssection along with JP 4.5.1 software components for installation.Method 2: Using the DeepStream tar package
Download the DeepStream 5.1 Jetson tar package
deepstream_sdk_v5.1.0_jetson.tbz2, to the Jetson device.Enter the following commands to extract and install the DeepStream SDK:
$ sudo tar -xvf deepstream_sdk_v5.1.0_jetson.tbz2 -C / $ cd /opt/nvidia/deepstream/deepstream-5.1 $ sudo ./install.sh $ sudo ldconfig
Method 3: Using the DeepStream Debian package
Download the DeepStream 5.1 Jetson Debian package
deepstream-5.1_5.1.0-1_arm64.deb, to the Jetson device. Then enter the command:$ sudo apt-get install ./deepstream-5.1_5.1.0-1_arm64.deb
Note
If you install the DeepStream SDK Debian package using the
dpkgcommand, you must install the following packages before installing the debian package:libgstrtspserver-1.0-0libgstreamer-plugins-base1.0-dev
Method 4: Using the apt-server
Open the apt source configuration file in a text editor, using a command similar to
$ sudo vi /etc/apt/sources.list.d/nvidia-l4t-apt-source.listChange the repository name and download URL in the deb commands shown below:
deb https://repo.download.nvidia.com/jetson/common r32.5 mainSave and close the source configuration file.
Enter the commands:
$ sudo apt update $ sudo apt install deepstream-5.1
Method 5: Use Docker container DeepStream docker containers are available on NGC. See the Docker Containers section to learn about developing and deploying DeepStream using docker containers.
Run deepstream-app (the reference application)¶
Navigate to the samples directory on the development kit.
Enter the following command to run the reference application:
$ deepstream-app -c <path_to_config_file>
Where
<path_to_config_file>is the pathname of one of the reference application’s configuration files, found inconfigs/deepstream-app/. See Package Contents for a list of the available files.Note
You can find sample configuration files under
/opt/nvidia/deepstream/deepstream-5.1/samplesdirectory. Enter this command to see application usage:$ deepstream-app --help
To save TensorRT Engine/Plan file, run the following command:
$ sudo deepstream-app -c <path_to_config_file>
To show labels in 2D Tiled display view, expand the source of interest with mouse left-click on the source. To return to the tiled display, right-click anywhere in the window.
Keyboard selection of source is also supported. On the console where application is running, press the
zkey followed by the desired row index (0 to 9), then the column index (0 to 9) to expand the source. To restore 2D Tiled display view, presszagain.
Boost the clocks¶
After you have installed DeepStream SDK, run these commands on the Jetson device to boost the clocks:
$ sudo nvpmodel -m 0
$ sudo jetson_clocks
Run precompiled sample applications¶
Navigate to the chosen application directory inside
sources/apps/sample_apps.Follow the directory’s README file to run the application.
Note
If the application encounters errors and cannot create Gst elements, remove the GStreamer cache, then try again. To remove the GStreamer cache, enter this command:
$ rm ${HOME}/.cache/gstreamer-1.0/registry.aarch64.binWhen the application is run for a model which does not have an existing engine file, it may take up to a few minutes (depending on the platform and the model) for the file generation and the application launch. For later runs, these generated engine files can be reused for faster loading.
dGPU Setup for Ubuntu¶
This section explains how to prepare an Ubuntu x86_64 system with NVIDIA dGPU devices before installing the DeepStream SDK.
Note
This document uses the term dGPU (“discrete GPU”) to refer to NVIDIA GPU expansion card products such as NVIDIA Tesla® T4 and P4, NVIDIA GeForce® GTX 1080, and NVIDIA GeForce® RTX 2080. This version of DeepStream SDK runs on specific dGPU products on x86_64 platforms supported by NVIDIA driver 460.32 and NVIDIA TensorRT™ 7.2 and later versions.
You must install the following components:
Ubuntu 18.04
GStreamer 1.14.1
NVIDIA driver 460.32
CUDA 11.1
TensorRT 7.2.X
Remove all previous DeepStream installations¶
Enter the following commands to remove all previous DeepStream 3.0 or prior installations:
$ sudo rm -rf /usr/local/deepstream /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstnv* /usr/bin/deepstream* /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libnvdsgst*
/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream*
/opt/nvidia/deepstream/deepstream*
$ sudo rm -rf /usr/lib/x86_64-linux-gnu/libv41/plugins/libcuvidv4l2_plugin.so
To remove DeepStream 4.0 or later installations:
Open the
uninstall.shfile in/opt/nvidia/deepstream/deepstream/Set PREV_DS_VER as 4.0
Run the following script as
sudo:./uninstall.sh
Install Dependencies¶
Enter the following commands to install the necessary packages before installing the DeepStream SDK:
$ sudo apt install \
libssl1.0.0 \
libgstreamer1.0-0 \
gstreamer1.0-tools \
gstreamer1.0-plugins-good \
gstreamer1.0-plugins-bad \
gstreamer1.0-plugins-ugly \
gstreamer1.0-libav \
libgstrtspserver-1.0-0 \
libjansson4
Install NVIDIA driver 460.32¶
Download and install NVIDIA driver 460.32 from NVIDIA Unix drivers page at: https://www.nvidia.com/download/driverResults.aspx/168347/en-us
Run the following commands:
$chmod 755 NVIDIA-Linux-x86_64-460.32.03.run $./ NVIDIA-Linux-x86_64-460.32.03.run
Install CUDA Toolkit 11.1¶
Download and install CUDA Toolkit 11.1 from NVIDIA Developer Center at: https://developer.nvidia.com/cuda-downloads
Install TensorRT 7.2.X¶
Download and install TensorRT 7.2 from the NVIDIA Developer Center: https://developer.nvidia.com/nvidia-tensorrt-download
Install librdkafka (to enable Kafka protocol adaptor for message broker)¶
Clone the librdkafka repository from GitHub:
$ git clone https://github.com/edenhill/librdkafka.git
Configure and build the library:
$ cd librdkafka $ git reset --hard 7101c2310341ab3f4675fc565f64f0967e135a6a ./configure $ make $ sudo make install
Copy the generated libraries to the deepstream directory:
$ sudo mkdir -p /opt/nvidia/deepstream/deepstream-5.1/lib $ sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream-5.1/lib
Install the DeepStream SDK¶
Method 1: Using the DeepStream Debian package
Download the DeepStream 5.1 dGPU Debian package
deepstream-5.1_5.1.0-1_amd64.deb: https://developer.nvidia.com/deepstream-51-510-1-amd64debEnter the command:
$ sudo apt-get install ./deepstream-5.1_5.1.0-1_amd64.deb
Method 2: Download the DeepStream tar package: https://developer.nvidia.com/deepstream-sdk-v510-x86-64tbz2
Navigate to the location of the downloaded DeepStream package to extract and install the DeepStream SDK:
$ sudo tar -xvf deepstream_sdk_v5.1.0_x86_64.tbz2 -C / $ cd /opt/nvidia/deepstream/deepstream-5.1/ $ sudo ./install.sh $ sudo ldconfig
Method 3: Using the apt-server
Add the following x86_64 repository in the source list of the machine.
Open the apt source configuration file in a text editor, for example:
$ sudo vi /etc/apt/sources.list.d/nvidia-apt-source.listConfirm repository name and download URL in the deb commands:
deb https://repo.download.nvidia.com/jetson/x86_64 bionic r32.5Save and close the source configuration file.
Install the public key of the x86_64 repository of the APT server:
$ sudo apt-key adv --fetch-key https://repo.download.nvidia.com/jetson/jetson-ota-public.asc
Enter the commands:
$ sudo apt update $ sudo apt install deepstream-5.1
Method 4: Use Docker container DeepStream docker containers are available on NGC. See the Docker Containers section to learn about developing and deploying DeepStream using docker containers.
Run the deepstream-app (the reference application)¶
Go to the samples directory and enter this command:
$ deepstream-app -c <path_to_config_file>
Where
<path_to_config_file>is the pathname of one of the reference application’s configuration files, found inconfigs/deepstream-app. See Package Contents for a list of the available files.Note
To dump engine file, run the following command:
$ sudo deepstream-app -c <path_to_config_file>
You can find sample configuration files under
/opt/nvidia/deepstream/deepstream-5.1/samplesdirectory. Enter this command to see application usage:$ deepstream-app --help
To show labels in 2D tiled display view, expand the source of interest with a mouse left-click on the source. To return to the tiled display, right-click anywhere in the window.
Keyboard selection of source is also supported. On the console where application is running, press the
zkey followed by the desired row index (0 to 9), then the column index (0 to 9) to expand the source. To restore the 2D Tiled display view, presszagain.
Run precompiled sample applications¶
Navigate to the chosen application directory inside
sources/apps/sample_apps.Follow that directory’s README file to run the application.
Note
If the application encounters errors and cannot create Gst elements, remove the GStreamer cache, then try again. To remove the GStreamer cache, enter this command:
$ rm ${HOME}/.cache/gstreamer-1.0/registry.x86_64.binWhen the application is run for a model which does not have an existing engine file, it may take up to a few minutes (depending on the platform and the model) for the file generation and application launch. For later runs, these generated engine files can be reused for faster loading.
dGPU Setup for RedHat Enterprise Linux (RHEL)¶
This section explains how to prepare an RHEL system with NVIDIA dGPU devices before installing the DeepStream SDK.
Note
This document uses the term dGPU (“discrete GPU”) to refer to NVIDIA GPU expansion card products such as NVIDIA Tesla T4 and P4, NVIDIA GeForce GTX 1080, and NVIDIA GeForce RTX 2080. This version of DeepStream SDK runs on specific dGPU products on x86_64 platforms supported by NVIDIA driver 460+ and NVIDIA® TensorRT™ 7.2 and later versions.
You must install the following components:
RHEL 8.x
GStreamer 1.14.1
NVIDIA driver 460.32 (https://www.nvidia.com/download/driverResults.aspx/168347/en-us)
CUDA 11.1
TensorRT 7.2.X
Remove all previous DeepStream installations¶
To remove DeepStream 4.0 or later installations:
Open the uninstall.sh file which will be present in
/opt/nvidia/deepstream/deepstream/Set
PREV_DS_VERas4.0Run the following script as sudo:
./uninstall.sh
Install Dependencies¶
Enter the following command to install the necessary packages before installing the DeepStream SDK:
$ yum install \
gstreamer1 \
gstreamer1-plugins-base \
gstreamer1-plugins-good \
gstreamer1-plugins-bad-free \
gstreamer1-plugins-ugly-free \
gstreamer1-rtsp-server \
gstreamer1-svt-av1 \
json-glib \
openssl \
libuuid \
gstreamer1-plugins-base-devel \
json-glib-devel \
opencv-devel \
jansson-devel \
openssl-devel \
libuuid-devel
gst-rtsp-server:
gst-rtsp-server-devel package is not available for RHEL which is required to compile deepstream-app.
Download sources from https://gstreamer.freedesktop.org/src/gst-rtsp-server/gst-rtsp-server-1.14.5.tar.xz:
$ ./configure
$ make
$ make install
$ sudo cp -r /usr/local/include/gstreamer-1.0/gst/rtsp-server/ /usr/include/gstreamer-1.0/gst/
Note
Packages required for RHEL 8.x are also mentioned in README.rhel in DeepStream package.
Install NVIDIA driver 460.32¶
Download and install NVIDIA driver 460.32 from NVIDIA unix drivers page at https://www.nvidia.com/download/driverResults.aspx/168347/en-us
Run the following commands:
$chmod 755 NVIDIA-Linux-x86_64-460.32.03.run $./ NVIDIA-Linux-x86_64-460.32.03.run
Install CUDA Toolkit 11.1¶
Download and install CUDA Toolkit 11.1 from NVIDIA Developer Center at: https://developer.nvidia.com/cuda-downloads
Install TensorRT 7.2.X¶
Download and install TensorRT 7.2.X from the NVIDIA Developer Center: https://developer.nvidia.com/nvidia-tensorrt-download
Install librdkafka (to enable Kafka protocol adaptor for message broker)¶
Clone the librdkafka repository from GitHub:
$ git clone https://github.com/edenhill/librdkafka.git
Configure and build the library:
$ cd librdkafka $ git reset --hard 7101c2310341ab3f4675fc565f64f0967e135a6a ./configure $ make $ sudo make install
Copy the generated libraries to the deepstream directory:
$ sudo mkdir -p /opt/nvidia/deepstream/deepstream-5.1/lib $ sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream-5.1/lib
Install the DeepStream SDK¶
Method 1: Download the DeepStream tar package: https://developer.nvidia.com/deepstream-sdk-v510-x86-64tbz2
Navigate to the location to which the DeepStream package was downloaded and extract and install the DeepStream SDK:
$ sudo tar -xvf deepstream_sdk_v5.1.0_x86_64.tbz2 -C / $ cd /opt/nvidia/deepstream/deepstream-5.1/ $ sudo ./install.sh $ sudo ldconfig
Method 2: Use Docker container
DeepStream docker containers are available on NGC. See the Docker Containers section to learn about developing and deploying DeepStream using docker containers.
Run the deepstream-app (the reference application)¶
Go to the samples directory and enter this command:
$ deepstream-app -c <path_to_config_file>
Where
<path_to_config_file>is the pathname of one of the reference application’s configuration files, found inconfigs/deepstream-app. See Package Contents for a list of the available files.Note
To dump engine file run the following command:
$ sudo deepstream-app -c <path_to_config_file>
You can find sample configuration files in the
/opt/nvidia/deepstream/deepstream-5.1/samplesdirectory. Enter this command to see application usage:$ deepstream-app --help
To show labels in 2D tiled display view, expand the source of interest with a mouse left-click on the source. To return to the tiled display, right-click anywhere in the window.
Keyboard selection of source is also supported. On the console where application is running, press the
zkey followed by the desired row index (0 to 9), then the column index (0 to 9) to expand the source. To restore the 2D Tiled display view, presszagain.
Run precompiled sample applications¶
Navigate to the chosen application directory inside
sources/apps/sample_apps.Follow that directory’s README file to run the application.
Note
If the application encounters errors and cannot create Gst elements, remove the GStreamer cache, then try again. To remove the GStreamer cache, enter this command:
$ rm ${HOME}/.cache/gstreamer-1.0/registry.x86_64.binWhen the application is run for a model which does not have an existing engine file, it may take up to a few minutes (depending on the platform and the model) for the file generation and application launch. For later runs, these generated engine files can be reused for faster loading.
Running without an X server¶
The default configuration files provided with the SDK have the EGL based nveglglessink as the default renderer (indicated by type=2 in the [sink] groups). The renderer requires a running X server and fails without one.
In case of absence of an X server, DeepStream reference applications provide an alternate functionality of streaming the output over RTSP. This can be enabled by adding an RTSP out sink group in the configuration file. Refer to [sink2] group in source30_1080p_dec_infer-resnet_tiled_display_int8.txt file for an example. Don’t forget to disable the nveglglessink renderer by setting enable=0 for the corresponding sink group.
Platform and OS Compatibility¶
The following table provides information about platform and operating system compatibility in the current and earlier versions of DeepStream.
DS release |
DS 1.0 |
DS 1.5 |
DS 2.0 |
DS 3.0 |
DS 4.0.1 (Unified) |
DS 4.0.2 (Unified) |
DS 5.0 GA, 5.0.1, 5.1 (Unified) |
|---|---|---|---|---|---|---|---|
Jetson platforms |
TX2, TX1 |
TX2, TX1 |
Not supported |
AGX Xavier |
Nano, AGX Xavier, TX2, TX1 |
Nano, AGX Xavier, TX2, TX1 |
Nano, AGX Xavier, TX2, TX1, Jetson NX |
OS |
L4T Ubuntu 16.04 |
L4T Ubuntu 16.04 |
Not supported |
L4T Ubuntu 18.04/16.04 |
L4T Ubuntu 18.04 |
L4T Ubuntu 18.04 |
L4T Ubuntu 18.04 |
JetPack release |
3.1 |
3.2 |
Not supported |
4.1.1 |
4.2.1 |
4.3 |
4.4 GA (4.5.1 GA for DS 5.1) |
L4T release |
28.1 |
28.2 |
Not supported |
31.1 |
32.2 |
32.3.1 |
32.4.3 (32.5.1 for DS 5.1) |
CUDA release |
CUDA 8.0 |
CUDA 9.0 |
Not supported |
CUDA 10.0 |
CUDA 10.0 |
CUDA 10.0 |
CUDA 10.2 |
cuDNN release |
cuDNN 6.0 |
cuDNN 7.0.5 |
Not supported |
cuDNN 7.3 |
cuDNN 7.5.1 |
cuDNN 7.6.3 |
cuDNN 8.0.0.x |
TensorRT release |
TRT 2.1 |
TRT 3.0 |
Not supported |
TRT 5.0 |
TRT 5.1.6 |
TRT 6.0.1 |
TRT 7.1.3 |
OpenCV release |
OpenCV 2.4.13 |
OpenCV 3.3.1 |
Not supported |
OpenCV 3.3.1 |
OpenCV 3.3.1 |
OpenCV 4.1 |
OpenCV 4.1.1 |
VisionWorks |
VisionWorks 1.6 |
VisionWorks 1.6 |
Not supported |
VisionWorks 1.6 |
VisionWorks 1.6 |
VisionWorks 1.6 |
VisionWorks 1.6 |
GStreamer |
GStreamer 1.8.3 |
GStreamer 1.8.3 |
Not supported |
GStreamer 1.8.3 |
GStreamer 1.14.1 |
GStreamer 1.14.1 |
GStreamer 1.14.1 |
Docker image |
Not available |
Not available |
Not available |
Not available |
deepstream-l4t:4.0 |
deepstream-l4t:4.0.2 |
deepstream-l4t:5.0, deepstream-l4t:5.0.1, deepstream-l4t:5.1 |
DS release |
DS 1.0 |
DS 1.5 |
DS 2.0 |
DS 3.0 |
DS 4.0.1 (Unified) |
DS 4.0.2 (Unified) |
DS 5.0 GA, 5.0.1 (Unified), 5.1 |
|---|---|---|---|---|---|---|---|
GPU platforms |
P4, P40 |
P4, P40 |
P4, P40 |
P4, P40, V100, T4 |
P4, T4, V100 |
P4, T4, V100 |
P4, T4, V100, GA100 (DS 5.1) |
OS |
Ubuntu 16.04 |
Ubuntu 16.04 |
Ubuntu 16.04 |
Ubuntu 16.04 |
Ubuntu 18.04 |
Ubuntu 18.04 |
Ubuntu 18.04 RHEL 8.x |
GCC |
GCC 5.4 |
GCC 5.4 |
GCC 5.4 |
GCC 5.4 |
GCC 7.3.0 |
GCC 7.3.0 |
GCC 7.3.0 |
CUDA release |
CUDA 8.0 |
CUDA 9.0 |
CUDA 9.2 |
CUDA 10.0 |
CUDA 10.1 |
CUDA 10.1 |
CUDA 10.2 ( Cuda 11.1 for DS 5.1) |
cuDNN release |
cuDNN 6.0 |
cuDNN 7.0 |
cuDNN 7.1 |
cuDNN 7.3 |
cuDNN 7.5.0+ |
cuDNN 7.6.5+ |
cuDNN 7.6.5+ (CuDNN 8.0+ for DS 5.1) |
TRT release |
TRT 2.1 |
TRT 3.0 |
TRT 4.0 |
TRT 5.0 |
TRT 5.1.5 |
TRT 6.0.1 |
TRT 7.0.0 (TRT 7.2.X for DS 5.1) |
Display Driver |
R375 |
R384 |
R396+ |
R410+ |
R418+ |
R418+ |
R450.51 (R460.32 for DS 5.1) |
VideoSDK release |
SDK 7.1 |
SDK 7.9 |
SDK 7.9 |
SDK 8.2 |
SDK 9.0 |
SDK 9.0 |
SDK 9.1 |
OFSDK release |
Not available |
Not available |
Not available |
Not available |
1.0.10 |
1.0.10 |
1.0.10 |
GStreamer release |
Not available |
GStreamer 1.8.3 |
GStreamer 1.8.3 |
GStreamer 1.8.3 |
GStreamer 1.14.1 |
GStreamer 1.14.1 |
GStreamer 1.14.1 |
OpenCV release |
Not available |
OpenCV 2.4.13 |
OpenCV 3.4.x |
OpenCV 3.4.x |
OpenCV 3.3.1 |
OpenCV 3.3.1 |
OpenCV 3.4.0 |
Docker image |
Not available |
Not available |
Not available |
deepstream:3.0 |
deepstream:4.0 |
deepstream:4.0.2 |
deepstream:5.0, deepstream:5.0.1, deepstream:5.1 |
DeepStream Triton Inference Server Usage Guidelines¶
dGPU¶
Pull the DeepStream Triton Inference Server docker
docker pull nvcr.io/nvidia/deepstream:5.1-21.02-triton
Start the docker
docker run --gpus all -it --rm -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY nvcr.io/nvidia/deepstream:5.1-21.02-triton
Jetson¶
The Triton Inference Server shared libraries come pre-installed as part of DeepStream on Jetson. No extra steps are required for installing the Triton Inference Server.
For both the platforms, to run the samples follow the steps in the Running the Triton Inference Server samples section of the README at: /opt/nvidia/deepstream/deepstream-5.1