Quick Start Guide

Note:

If you install the DeepStream SDK Debian package using the dpkg command, you must install these packages first, then install the DeepStream deb package:

• libgstrtspserver-1.0-0

• libgstreamer-plugins-base1.0-dev

Note:

For deb installation, you can find sample configuration files under /opt/nvidia/deepstream/deepstream-4.0/samples directory.

Enter this command to see application usage:

$ deepstream-app --help

Note:

If the application encounters errors and cannot create Gst elements, remove the GStreamer cache, then try again. To remove the GStreamer cache, enter this command:

$ rm ${HOME}/.cache/gstreamer-1.0/registry.aarch64.bin

Note:

This document uses the term dGPU (“discrete GPU”) to refer to NVIDIA GPU expansion card products such as NVIDIA® Tesla® T4 and P4, NVIDIA® GeForce® GTX 1080, and NVIDIA® GeForce® RTX 2080.

This version of DeepStream SDK runs on specific dGPU products on x86_64 platforms supported by NVIDIA driver 418+ and NVIDIA® TensorRT™ 5.1 and later versions.

Note:

If you install the DeepStream SDK Debian package using the dpkg command, you must install these packages first, then install the DeepStream deb package:

• libgstrtspserver-1.0-0

• libgstreamer-plugins-base1.0-dev

Note:

If you installed using the DeepStream Debian package, you can find sample configuration files under /opt/nvidia/deepstream/deepstream-4.0/samples directory.

Enter this command to see application usage:

$ deepstream-app --help

Note:

If the application encounters errors and cannot create Gst elements, remove the GStreamer cache, then try again. To remove the GStreamer cache, enter this command:

$ rm ${HOME}/.cache/gstreamer-1.0/registry.x86_64.bin

Plugin or library	Path inside sources directory	Description
DsExample GStreamer plugin	gst-plugins/gst-dsexample	Template plugin for integrating custom algorithms into DeepStream SDK graph.
GStreamer Gst-nvmsgconv plugin	gst-plugins/gst-nvmsgconv	Source code for the GStreamer Gst-nvmsgconv plugin for converting metadata to schema format.
GStreamer Gst-nvmsgbroker plugin	gst-plugins/gst-nvmsgbroker	Source code for the GStreamer Gst-nvmsgbroker plugin for sending data to the server.
GStreamer Gst-nvinfer plugin	gst-plugins/gst-nvinfer	Source code for the GStreamer Gst-nvinfer plugin for inference.
NvDsInfer library	libs/nvdsinfer	Source code for the NvDsInfer library, used by the Gst-nvinfer GStreamer plugin.
NvMsgConv library	libs/nvmsgsconv	Source code for the NvMsgConv library, required by the Gst-nvmsgconv GStreamer plugin.
Kafka protocol adapter	libs/kafka_protocol_adapter	Protocol adapter for Kafka.
nvdsinfer_customparser	libs/nvdsinfer_customparser	Custom model output parsing example for detectors and classifiers.
Gst-v4l2	See the note below *	Source code for v4l2 codecs.
* Gst-v4l2 sources are not present in DeepStream package. To download, follow these steps: 1. Go to: https://developer.nvidia.com/embedded/downloads. 2. In the “Search filter” field, enter “L4T sources.” 3. Select the appropriate item for L4T Release 32.2.1. 4. Download the file and untar it, yielding a .tbz2 file. 5. Expand the .tbz2 file. Gst-v4l2 source files are in gst-nvvideo4linux2_src.tbz2.

Reference test application	Path inside sources directory	Description
Simple test application 1	apps/sample_apps/deepstream-test1	Simple example of how to use DeepStream elements for a single H.264 stream: filesrc → decode → nvstreammux → nvinfer (primary detector) → nvosd → renderer.
Simple test application 2	apps/sample_apps/deepstream-test2	Simple example of how to use DeepStream elements for a single H.264 stream: filesrc → decode → nvstreammux → nvinfer (primary detector) → nvtracker → nvinfer (secondary classifier) → nvosd → renderer.
Simple test application 3	apps/sample_apps/deepstream-test3	Builds on deepstream-test1 (simple test application 1) to demonstrate how to: • Use multiple sources in the pipeline • Use a uridecodebin to accept any type of input (e.g. RTSP/File), any GStreamer supported container format, and any codec • Configure Gst-nvstreammux to generate a batch of frames and infer on it for better resource utilization • Extract the stream metadata, which contains useful information about the frames in the batched buffer
Simple test application 4	apps/sample_apps/deepstream-test4	Builds on deepstream-test1 for a single H.264 stream: filesrc, decode, nvstreammux, nvinfer, nvosd, renderer to demonstrate how to: • Use the Gst-nvmsgconv and Gst-nvmsgbroker plugins in the pipeline • Create NVDS_META_EVENT_MSG type metadata and attach it to the buffer • Use NVDS_META_EVENT_MSG for different types of objects, e.g. vehicle and person • Implement “copy” and “free” functions for use if metadata is extended through the extMsg field
Simple test application 5	apps/sample_apps/deepstream-test5	Builds on top of deepstream-app. Demonstrates: • Use of Gst-nvmsgconv and Gst-nvmsgbroker plugins in the pipeline for multistream • How to configure Gst-nvmsgbroker plugin from the config file as a sink plugin (for KAFKA, Azure, etc.) • How to handle the RTCP sender reports from RTSP servers or cameras and translate the Gst Buffer PTS to a UTC timestamp. For more details refer the RTCP Sender Report callback function test5_rtcp_sender_report_callback() registration and usage in deepstream_test5_app_main.c. GStreamer callback registration with rtpmanager element’s “handle-sync” signal is documented in apps-common/src/deepstream_source_bin.c.
AMQP protocol test application	libs/amqp_protocol_adaptor	Test application for AMQP.
Azure MQTT test application	libs/azure_protocol_adaptor	Test application for Azure MQTT.
DeepStream reference application	apps/sample_apps/deepstream-app	Source code for the DeepStream reference application.
UFF SSD detector	sources/objectDetector_SSD	Configuration files and custom library implementation for the SSD detector model.
Faster RCNN detector	sources/objectDetector_FasterRCNN	Configuration files and custom library implementation for the FasterRCNN model.
Yolo detector	sources/objectDetector_Yolo	Configuration files and custom library implementation for the Yolo models, currently Yolo v2, v2 tiny, v3, and v3 tiny.
Dewarper example	apps/sample_apps/deepstream-dewarper-test	Demonstrates dewarper functionality for single or multiple 360 degree camera streams. Reads camera calibration parameters from a CSV file and renders aisle and spot surfaces on the display.
Optical flow example	apps/sample_apps/deepstream-nvof-test	Demonstrates optical flow functionality for single or multiple streams. This example uses two GStreamer plugins (Gst-nvof and Gst-nvofvisual). The Gst-nvof element generates the MV (motion vector) data and attaches it as user metadata. The Gst-nvofvisual element visualizes the MV data using a predefined color wheel matrix.
Custom meta data example	apps/sample_apps/deepstream-user-metadata-test	Demonstrates how to add custom or user-specfic metadata to any component of DeepStream. The test code attaches a 16-byte array filled with user data to the chosen component. The data is retrieved in another component.
MJPEG and JPEG decoder and inferencing example	apps/sample_apps/deepstream-image-decode-test	Builds on deepstream-test3 to demonstrate image decoding instead of video. This example uses a custom decode bin so the MJPEG codec can be used as input.
Image/Video segmentation example	apps/sample_apps/deepstream-segmentation-test	Demonstrates segmentation of multi-stream video or images using a semantic or industrial neural network, and rendering output to a display.
Handling metadata before Gst-nvstreammux	apps/sample_apps/deepstream-gst-metadata-test	Demonstrates how to set metadata before the Gst-nvstreammux plugin in the DeepStream pipeline, and how to access it after Gst-nvstreammux.
Gst-nvinfer tensor meta flow example	apps/sample_apps/deepstream-infer-tensor-meta-app	Demonstrates how to flow and access nvinfer tensor output as metadata.
Performance demo	apps/sample_apps/deepstream-perf-demo	Performs single channel cascaded inferencing and object tracking sequentially on all streams in a directory.

Stream	Type of Stream
sample_1080p_h264.mp4	H264 containerized stream
sample_1080p_h265.mp4	H265 containerized stream
sample_720p.h264	H264 elementary stream
sample_720p.jpg	JPEG image
sample_720p.mjpeg	MJPEG stream
sample_cam6.mp4	H264 containerized stream (360D camera stream)
sample_industrial.jpg	JPEG image

DeepStream Reference Application
Model	Model Type	Number of Classes	Resolution
Primary Detector	Resnet10	4	640 × 368
Secondary Car Color Classifier	Resnet18	12	224 × 224
Secondary Car Make Classifier	Resnet18	6	224 × 224
Secondary Vehicle Type Classifier	Resnet18	20	224 × 224

Segmentation example
Model	Model Type	Number of Classes	Resolution
Industrial	Resnet18 + UNet	1	512 x 512
Semantic	Resnet18 + UNet	4	512 x 512