Installation

NVIDIA^® DeepStream Software Development Kit (SDK) is an accelerated AI framework to build intelligent video analytics (IVA) pipelines. DeepStream runs on NVIDIA^® T4, NVIDIA® Hopper, NVIDIA^® Ampere, NVIDIA^® ADA and platforms such as NVIDIA^® Jetson AGX Orin™, NVIDIA^® Jetson Orin™ NX, NVIDIA^® Jetson Orin™ Nano. For dGPU platforms Enterprise GPUs are highly recommended for deployments that are expected to run 24x7. Gaming GPUs are not designed to perform in such type of environments.

Jetson Setup

This section explains how to prepare a Jetson device before installing the DeepStream SDK.

Note

Steps to install DeepStream SDK locally, assume that ~/.local/bin/ has been added to the ~/.bashrc or ~/.profile

Install Jetson SDK components

Download NVIDIA SDK Manager from https://developer.nvidia.com/embedded/jetpack. You will use this to install JetPack 6.0 GA (corresponding to L4T 36.3 release)

NVIDIA SDK Manager is a graphical application which flashes and installs the JetPack packages.
The flashing procedure takes approximately 10-30 minutes, depending on the host system.

Note

If you are using Jetson Orin nano developer kit, you can download the SD card image from https://developer.nvidia.com/embedded/jetpack. This comes packaged with CUDA, TensorRT and cuDNN.

Prerequisites

Note

Sometimes while running gstreamer pipeline or sample apps, user can encounter error : GLib (gthread-posix.c): Unexpected error from C library during 'pthread_setspecific': Invalid argument. Aborting. The issue is caused because of a bug in glib 2.0-2.72 version which comes with ubuntu22.04 by default. The issue is addressed in glib2.76 and its installation is required to fix the issue (https://github.com/GNOME/glib/tree/2.76.6).

Migrate glib to newer version

In order to migrate to newer glib version (e.g. 2.76.6) follow below steps:

Prerequisites: Install below packages:
```
pip3 install meson
pip3 install ninja
```

Note

On baremetal, use sudo pip3 install meson and sudo pip3 install ninja for root permissions

Compilation and installation steps:

$ git clone https://github.com/GNOME/glib.git
$ cd glib
$ git checkout <glib-version-branch>
# e.g. 2.76.6
$ meson build --prefix=/usr
$ ninja -C build/
$ cd build/
$ ninja install

Note

On baremetal, use sudo ninja install to install with root permissions

Check and confirm the newly installed glib version:

pkg-config --modversion glib-2.0

Install Dependencies

Install prerequisite packages

Enter the following commands to install the prerequisite packages:

$ sudo apt install \
libssl3 \
libssl-dev \
libgstreamer1.0-0 \
gstreamer1.0-tools \
gstreamer1.0-plugins-good \
gstreamer1.0-plugins-bad \
gstreamer1.0-plugins-ugly \
gstreamer1.0-libav \
libgstreamer-plugins-base1.0-dev \
libgstrtspserver-1.0-0 \
libjansson4 \
libyaml-cpp-dev

Note

Sometimes with RTSP streams the application gets stuck on reaching EOS. This is because of an issue in rtpjitterbuffer component. To fix this issue,a script “update_rtpmanager.sh” at /opt/nvidia/deepstream/deepstream/ has been provided with required details to update gstrtpmanager library. The script should be executed once above mentioned packages are installed as prerequisite.

Install librdkafka (to enable Kafka protocol adaptor for message broker)

Clone the librdkafka repository from GitHub:

$ git clone https://github.com/confluentinc/librdkafka.git

Configure and build the library:

$ cd librdkafka
$ git checkout tags/v2.2.0
$ ./configure --enable-ssl
$ make
$ sudo make install

Copy the generated libraries to the deepstream directory:

$ sudo mkdir -p /opt/nvidia/deepstream/deepstream/lib
$ sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream/lib
$ sudo ldconfig

Install latest NVIDIA BSP packages

Installation of JetPack 6.0 GA will ensure that latest NVIDIA BSP packages are installed.

Install the DeepStream SDK

Method 1: Using SDK Manager

Select DeepStreamSDK from the Additional SDKs section along with JP 6.0 GA software components for installation.
Method 2: Using the DeepStream tar package: https://catalog.ngc.nvidia.com/orgs/nvidia/resources/deepstream
1. Download the DeepStream 7.0 Jetson tar package deepstream_sdk_v7.0.0_jetson.tbz2 to the Jetson device.
2. Enter the following commands to extract and install the DeepStream SDK:
  $ sudo tar -xvf deepstream_sdk_v7.0.0_jetson.tbz2 -C / $ cd /opt/nvidia/deepstream/deepstream-7.0 $ sudo ./install.sh $ sudo ldconfig
Method 3: Using the DeepStream Debian package: https://catalog.ngc.nvidia.com/orgs/nvidia/resources/deepstream

Download the DeepStream 7.0 Jetson Debian package deepstream-7.0_7.0.0-1_arm64.deb to the Jetson device. Enter the following command:
```
$ sudo apt-get install ./deepstream-7.0_7.0.0-1_arm64.deb
```
Method 4: Use Docker containers

DeepStream docker containers are available on NGC. See the Docker Containers section to learn about developing and deploying DeepStream using docker containers.

Note

Verification: Once DeepStream SDK installation is successful, refer to Expected output (deepstream-app) for the expected output.

dGPU Setup for Ubuntu

This section explains how to prepare an Ubuntu x86_64 system with NVIDIA dGPU devices before installing the DeepStream SDK.

Note

Steps to install DeepStream SDK locally, assume that ~/.local/bin/ has been added to the ~/.bashrc or ~/.profile

Note

This document uses the term dGPU (“discrete GPU”) to refer to NVIDIA GPU expansion card products such as NVIDIA Tesla^® T4, NVIDIA® Hopper, NVIDIA^® Ampere, NVIDIA^® ADA, NVIDIA GeForce^® GTX 1080, NVIDIA GeForce^® RTX 2080, NVIDIA GeForce^® RTX 3080, NVIDIA GeForce^® RTX 4080 and GeForce^®/NVIDIA RTX/QUADRO. This version of DeepStream SDK runs on GPUs supported by NVIDIA driver 535.161.08 and NVIDIA TensorRT™ 8.6.1.6 and later versions.

Prerequisites

You must install the following components:

Ubuntu 22.04
GStreamer 1.20.3
NVIDIA driver 535.161.08
CUDA 12.2
TensorRT 8.6.1.6

Remove all previous DeepStream installations

Enter the following commands to remove all previous DeepStream 3.0 or prior installations:

$ sudo rm -rf /usr/local/deepstream /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstnv* /usr/bin/deepstream* /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libnvdsgst*
/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream*
/opt/nvidia/deepstream/deepstream*
$ sudo rm -rf /usr/lib/x86_64-linux-gnu/libv41/plugins/libcuvidv4l2_plugin.so

To remove DeepStream 4.0 or later installations:

Open the uninstall.sh file in /opt/nvidia/deepstream/deepstream/
Run the following script as sudo ./uninstall.sh

Note

Sometimes while running gstreamer pipeline or sample apps, user can encounter error : GLib (gthread-posix.c): Unexpected error from C library during 'pthread_setspecific': Invalid argument. Aborting. The issue is caused because of a bug in glib 2.0-2.72 version which comes with ubuntu22.04 by default. The issue is addressed in glib2.76 and its installation is required to fix the issue (https://github.com/GNOME/glib/tree/2.76.6).

Migrate glib to newer version

In order to migrate to newer glib version (e.g. 2.76.6) follow below steps:

Prerequisites: Install below packages:
```
pip3 install meson
pip3 install ninja
```

Note

On baremetal x86, use sudo pip3 install meson and sudo pip3 install ninja for root permissions

Compilation and installation steps:

$ git clone https://github.com/GNOME/glib.git
$ cd glib
$ git checkout <glib-version-branch>
# e.g. 2.76.6
$ meson build --prefix=/usr
$ ninja -C build/
$ cd build/
$ ninja install

Note

On baremetal x86, use sudo ninja install to install with root permissions

Check and confirm the newly installed glib version:

pkg-config --modversion glib-2.0

Install Dependencies

Install prerequisite packages

Enter the following commands to install the necessary packages before installing the DeepStream SDK:

$ sudo apt install \
libssl3 \
libssl-dev \
libgles2-mesa-dev \
libgstreamer1.0-0 \
gstreamer1.0-tools \
gstreamer1.0-plugins-good \
gstreamer1.0-plugins-bad \
gstreamer1.0-plugins-ugly \
gstreamer1.0-libav \
libgstreamer-plugins-base1.0-dev \
libgstrtspserver-1.0-0 \
libjansson4 \
libyaml-cpp-dev \
libjsoncpp-dev \
protobuf-compiler \
gcc \
make \
git \
python3

Note

Sometimes with RTSP streams the application gets stuck on reaching EOS. This is because of an issue in rtpjitterbuffer component. To fix this issue,a script “update_rtpmanager.sh” at /opt/nvidia/deepstream/deepstream/ has been provided with required details to update gstrtpmanager library. The script should be executed once above mentioned packages are installed as prerequisite.

Install CUDA Toolkit 12.2

Run the following commands (reference, https://developer.nvidia.com/cuda-downloads):

$ sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/3bf863cc.pub
$ sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/ /"
$ sudo apt-get update
$ sudo apt-get install cuda-toolkit-12-2

Note

If you observe following errors while CUDA installation, refer to https://developer.nvidia.com/blog/updating-the-cuda-linux-gpg-repository-key/.

W: GPG error: https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64 InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY A4B469963BF863CC
E: The repository 'https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64 InRelease' is no longer signed.

Install NVIDIA driver 535.161.08

Download and install using NVIDIA driver 535.161.08 from NVIDIA Unix drivers page at: https://www.nvidia.cn/Download/driverResults.aspx/222416/en-us/

Run the following commands:

$chmod 755 NVIDIA-Linux-x86_64-535.161.08.run
$sudo ./NVIDIA-Linux-x86_64-535.161.08.run --no-cc-version-check

Note

Ensure gdm, lightdm or Xorg service is stopped while installing nvidia driver

Use command : sudo service gdm stop sudo service lightdm stop sudo pkill -9 Xorg

To Run higher number of streams (200+) on Hopper, Ampere and Ada, follow below instructions:

$ sudo service display-manager stop
#Make sure no process is running on GPU i.e. Xorg or trition server etc
$ sudo pkill -9 Xorg
#Remove kernel modules
$ sudo rmmod nvidia_drm nvidia_modeset nvidia
#Load Modules with Regkeys
$ sudo modprobe nvidia NVreg_RegistryDwords="RMDebugOverridePerRunlistChannelRam = 1;RMIncreaseRsvdMemorySizeMB = 1024;RMDisableChIdIsolation = 0x1;RmGspFirmwareHeapSizeMB = 256"
$ sudo service display-manager start

Install TensorRT 8.6.1.6

Run the following command to install TensorRT 8.6.1.6:

sudo apt-get install --no-install-recommends libnvinfer-lean8=8.6.1.6-1+cuda12.0 libnvinfer-vc-plugin8=8.6.1.6-1+cuda12.0 \
libnvinfer-headers-dev=8.6.1.6-1+cuda12.0 libnvinfer-dev=8.6.1.6-1+cuda12.0 libnvinfer-headers-plugin-dev=8.6.1.6-1+cuda12.0 \
libnvinfer-plugin-dev=8.6.1.6-1+cuda12.0 libnvonnxparsers-dev=8.6.1.6-1+cuda12.0 libnvinfer-lean-dev=8.6.1.6-1+cuda12.0 \
libnvparsers-dev=8.6.1.6-1+cuda12.0 python3-libnvinfer-lean=8.6.1.6-1+cuda12.0 python3-libnvinfer-dispatch=8.6.1.6-1+cuda12.0 \
uff-converter-tf=8.6.1.6-1+cuda12.0 onnx-graphsurgeon=8.6.1.6-1+cuda12.0 libnvinfer-bin=8.6.1.6-1+cuda12.0 \
libnvinfer-dispatch-dev=8.6.1.6-1+cuda12.0 libnvinfer-dispatch8=8.6.1.6-1+cuda12.0 libnvonnxparsers-dev=8.6.1.6-1+cuda12.0 \
libnvonnxparsers8=8.6.1.6-1+cuda12.0 libnvinfer-vc-plugin-dev=8.6.1.6-1+cuda12.0 libnvinfer-samples=8.6.1.6-1+cuda12.0

Note

It is assumed, “deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/” has been added, as mentioned in CUDA Toolkit 12.2 installation step.

Install librdkafka (to enable Kafka protocol adaptor for message broker)

Clone the librdkafka repository from GitHub:

$ git clone https://github.com/confluentinc/librdkafka.git

Configure and build the library:

$ cd librdkafka
$ git checkout tags/v2.2.0
$ ./configure --enable-ssl
$ make
$ sudo make install

Copy the generated libraries to the deepstream directory:

$ sudo mkdir -p /opt/nvidia/deepstream/deepstream/lib
$ sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream/lib
$ sudo ldconfig

Install the DeepStream SDK

Method 1: Using the DeepStream Debian package

Download the DeepStream 7.0 dGPU Debian package deepstream-7.0_7.0.0-1_amd64.deb : https://catalog.ngc.nvidia.com/orgs/nvidia/resources/deepstream

Enter the command:
```
$ sudo apt-get install ./deepstream-7.0_7.0.0-1_amd64.deb
```
Method 2: Download the DeepStream tar package: https://catalog.ngc.nvidia.com/orgs/nvidia/resources/deepstream

Navigate to the location of the downloaded DeepStream package to extract and install the DeepStream SDK:
```
$ sudo tar -xvf deepstream_sdk_v7.0.0_x86_64.tbz2 -C /
$ cd /opt/nvidia/deepstream/deepstream-7.0/
$ sudo ./install.sh
$ sudo ldconfig
```
Method 3: Use Docker containers

DeepStream docker containers are available on NGC. See the Docker Containers section to learn about developing and deploying DeepStream using docker containers.

Note

Verification: Once DeepStream SDK installation is successful, refer to Expected output (deepstream-app) for the expected output.

dGPU on ARM Setup for Ubuntu (Alpha)

This section explains how to prepare an Ubuntu aarch64 system with NVIDIA dGPU devices before installing the DeepStream SDK.

Note

This document uses the term dGPU (“discrete GPU”) to refer to NVIDIA GPU expansion card products such as NVIDIA Tesla T4, NVIDIA Hopper, NVIDIA Ampere, NVIDIA® ADA, NVIDIA GeForce GTX 1080, NVIDIA GeForce RTX 2080, NVIDIA GeForce RTX 3080, NVIDIA GeForce RTX 4080 and GeForce/NVIDIA RTX/QUADRO. This version of DeepStream SDK runs on GPUs supported by NVIDIA driver 535.161.08 and NVIDIA TensorRT™ 8.6.1.6 and later versions.

You must install the following components:

NVIDIA driver 535.161.08
Docker
Nvidia Container Runtime

Install NVIDIA driver 535.161.08

Download and install using NVIDIA driver 535.161.08 from NVIDIA Unix drivers page at: https://www.nvidia.com/en-us/drivers/unix/linux-aarch64-archive/

Run the following commands:

$chmod 755 NVIDIA-Linux-aarch64-535.161.08.run
$sudo ./NVIDIA-Linux-aarch64-535.161.08.run --no-cc-version-check

Note

Ensure gdm, lightdm or Xorg service is stopped while installing nvidia driver

Use command : sudo service gdm stop sudo service lightdm stop sudo pkill -9 Xorg

To Run higher number of streams (200+) on Hopper, Ampere and Ada, follow below instructions:

$ sudo service display-manager stop
#Make sure no process is running on GPU i.e. Xorg or trition server etc
$ sudo pkill -9 Xorg
#Remove kernel modules
$ sudo rmmod nvidia_drm nvidia_modeset nvidia
#Load Modules with Regkeys
$ sudo modprobe nvidia NVreg_RegistryDwords="RMDebugOverridePerRunlistChannelRam = 1;RMIncreaseRsvdMemorySizeMB = 1024;RMDisableChIdIsolation = 0x1;RmGspFirmwareHeapSizeMB = 256"
$ sudo service display-manager start

Run dGPU on ARM Docker (SBSA)

Pull the DeepStream Triton Inference Server docker

docker pull nvcr.io/nvidia/deepstream:7.0-triton-arm-sbsa

Start the docker

sudo docker run -it --rm --runtime=nvidia --network=host -e NVIDIA_DRIVER_CAPABILITIES=compute,utility,video,graphics --gpus all --privileged -e DISPLAY=:0 -v /tmp/.X11-unix:/tmp/.X11-unix -v /etc/X11:/etc/X11 nvcr.io/nvidia/deepstream:7.0-triton-arm-sbsa

dGPU Setup for RedHat Enterprise Linux (RHEL)

DeepStream for RHEL is not supported in this release.

Platform and OS Compatibility

The following table provides information about platform and operating system compatibility in the current and earlier versions of DeepStream.

Jetson model Platform and OS Compatibility

DS release

DS 6.1

DS 6.1.1

DS 6.2

DS 6.3

DS 6.4

DS 7.0

Jetson platforms

AGX Xavier, Jetson NX, Jetson Orin

AGX Xavier, Jetson NX, Jetson Orin

Xavier AGX , Xavier NX, AGX Orin, Orin NX

Xavier AGX , Xavier NX, AGX Orin, Orin NX, Orin nano

AGX Orin, Orin NX, Orin nano

AGX Orin, Orin NX, Orin nano

OS

L4T Ubuntu 20.04

L4T Ubuntu 20.04

L4T Ubuntu 20.04

L4T Ubuntu 20.04

L4T Ubuntu 22.04

L4T Ubuntu 22.04

JetPack release

5.0.1 DP

5.0.2 GA Revision 1

5.1 GA

5.1.2 GA

6.0 DP

6.0 GA

L4T release

34.1.1

35.1

35.2.1

35.4

36.2

36.3

CUDA release

CUDA 11.4

CUDA 11.4

CUDA 11.4

CUDA 11.4

CUDA 12.2

CUDA 12.2

cuDNN release

cuDNN 8.3.2.49

cuDNN 8.4.1.50+

cuDNN 8.6.0.166+

cuDNN 8.6.0.166+

cuDNN 8.9.4.25+

cuDNN 8.9.4.25+

TensorRT release

TRT 8.4.0.11

TRT 8.4.1.5

TRT 8.5.2.2

TRT 8.5.2.2

TRT 8.6.2.3

TRT 8.6.2.3

OpenCV release

OpenCV 4.2.0

OpenCV 4.2.0

OpenCV 4.2.0

OpenCV 4.5.4

OpenCV 4.8.0

OpenCV 4.8.0

VisionWorks

NA

NA

NA

NA

NA

NA

GStreamer

GStreamer 1.16.2

GStreamer 1.16.2

GStreamer 1.16.3

GStreamer 1.16.3

GStreamer 1.20.3

GStreamer 1.20.3

Docker image

deepstream-l4t:6.1

deepstream-l4t:6.1.1

deepstream-l4t:6.2

deepstream-l4t:6.3

deepstream:6.4

deepstream:7.0

dGPU model Platform and OS Compatibility

DS release

DS 6.1

DS 6.1.1

DS 6.2

DS 6.3

DS 6.4

DS 7.0

GPU platforms

T4, V100, A2, A10, A30, A100, RTX Ampere (Ax000/RTX30x0)

T4, V100, A2, A10, A30, A100, RTX Ampere (Ax000/RTX30x0)

T4, V100, A2, A10, A30, A100, RTX Ampere (Ax000/RTX30x0), Hopper, ADA

T4, V100, A2, A10, A30, A100, RTX Ampere (Ax000/RTX30x0), Hopper, ADA

T4, V100, A2, A10, A30, A100, RTX Ampere (Ax000/RTX30x0), Hopper, ADA

T4, V100, A2, A10, A30, A100, RTX Ampere (Ax000/RTX30x0), Hopper, ADA

OS

Ubuntu 20.04

Ubuntu 20.04

Ubuntu 20.04

Ubuntu 20.04

Ubuntu 22.04

Ubuntu 22.04

GCC

GCC 9.4.0

GCC 9.4.0

GCC 9.4.0

GCC 9.4.0

GCC 11.4.0

GCC 11.4.0

CUDA release

CUDA 11.6.1

CUDA 11.7.1

CUDA 11.8

CUDA 12.1

CUDA 12.2

CUDA 12.2

cuDNN release

cuDNN 8.4.0.27

cuDNN 8.4.1.50+

cuDNN 8.7.0.84-1+

cuDNN 8.8.1.3-1+

cuDNN 8.9.4.25-1+

cuDNN 8.9.6.50-1+

TRT release

TRT 8.2.5.1

TRT 8.4.1.5

TRT 8.5.2.2

TRT 8.5.3.1

TRT 8.6.1.6

TRT 8.6.1.6

Display Driver

R510.47.03

R515.65.01

R525.85.12

R525.125.06

R535.104.12

R535.161.08

VideoSDK release

SDK 9.1

SDK 9.1

SDK 9.1

SDK 9.1

SDK 9.1

SDK 9.1

OFSDK release

2.0.23

2.0.23

2.0.23

2.0.23

2.0.23

2.0.23

GStreamer release

GStreamer 1.16.2

GStreamer 1.16.2

GStreamer 1.16.3

GStreamer 1.16.3

GStreamer 1.20.3

GStreamer 1.20.3

OpenCV release

OpenCV 4.2.0

OpenCV 4.2.0

OpenCV 4.2.0

OpenCV 4.2.0

OpenCV 4.5.4

OpenCV 4.5.4

Docker image

deepstream:6.1

deepstream:6.1.1

deepstream:6.2

deepstream:6.3

deepstream:6.4

deepstream:7.0

NVAIE release

NA

NA

NVAIE-3.x

NVAIE-3.x

NVAIE-4.x

NA

Note

By default, OpenCV has been deprecated. However, OpenCV can be enabled in plugins such as nvinfer (nvdsinfer) and dsexample (gst-dsexample) by setting WITH_OPENCV=1 in the Makefile of these components. Please refer component README for more instructions.

NA: “Not Applicable”

DeepStream 7.0 comes with Alpha support on ARM SBSA platform. This support is currently enabled by SBSA triton docker. SBSA category currently supports GH100.