Logo
DeepStream Version: 5.0

DeepStream Getting Started

  • Welcome to the DeepStream Documentation
    • NVIDIA DeepStream Overview
      • DeepStream Graph Architecture
      • DeepStream reference app
      • Getting started with building apps
      • DeepStream in Python
  • QuickStart Guide
    • Jetson Setup
      • Install Jetson SDK components
      • Install Dependencies
        • Install librdkafka (to enable Kafka protocol adaptor for message broker)
        • Install NVIDIA V4L2 GStreamer plugin
      • Install the DeepStream SDK
      • Run deepstream-app (the reference application)
        • Boost the clocks
      • Run precompiled sample applications
    • dGPU Setup for Ubuntu
      • Remove all previous DeepStream installations
      • Install Dependencies
        • Install NVIDIA driver 450.51
        • Install CUDA Toolkit 10.2
        • Install TensorRT 7.0
        • Install librdkafka (to enable Kafka protocol adaptor for message broker)
      • Install the DeepStream SDK
      • Run the deepstream-app (the reference application)
      • Run precompiled sample applications
    • dGPU Setup for RedHat Enterprise Linux (RHEL)
      • Remove all previous DeepStream installations
      • Install Dependencies
        • Install NVIDIA driver 450.51
        • Install CUDA Toolkit 10.2
        • Install TensorRT 7.0
        • Install librdkafka (to enable Kafka protocol adaptor for message broker)
      • Install the DeepStream SDK
      • Run the deepstream-app (the reference application)
      • Run precompiled sample applications
    • Running without an X server
    • Platform and OS Compatibility
    • DeepStream Triton Inference Server Usage Guidelines
      • dGPU
      • Jetson
  • Docker Containers
    • A Docker Container for dGPU
    • A Docker Container for Jetson

DeepStream Samples

  • C/C++ Sample Apps Source Details
    • Plugin and Library Source Details
  • Python Sample Apps Source Details
    • Python Bindings
      • Prerequisites
      • Running Sample Applications
      • Pipeline Construction
    • MetaData Access
      • Memory Management
      • Allocations
      • String Access
        • Setting String Fields
        • Reading String Fields
      • Casting
      • Callback Function Registration
      • Optimizations and Utilities
      • Image Data Access
    • Sample Application Source Details
  • DeepStream Reference Application - deepstream-app
    • Application Architecture
    • Reference Application Configuration
      • Expected Output for the DeepStream Reference Application (deepstream-app)
    • Configuration Groups
      • Application Group
      • Tiled-display Group
      • Source Group
      • Streammux Group
      • Primary GIE and Secondary GIE Group
      • Tracker Group
      • Message Converter Group
      • Message Consumer Group
      • OSD Group
      • Sink Group
      • Tests Group
      • NvDs-analytics Group
    • Application Tuning for DeepStream SDK
      • Performance Optimization
        • DeepStream best practices
        • Jetson optimization
        • Triton
        • Inference Throughput
      • Reducing Spurious Detections
  • DeepStream Reference Application - deepstream-test5 app
    • IoT Protocols supported and cloud configuration
    • Message consumer
    • Smart Record - Event based recording
    • OTA model update
      • Using the OTA functionality
  • DeepStream Reference Application on GitHub
    • Use case applications
    • AI models with DeepStream
    • DeepStream features sample
  • Sample Configurations and Streams
    • Contents of the package
      • Scripts included along with package
  • Implementing a Custom GStreamer Plugin with OpenCV Integration Example
    • Description of the Sample Plugin: gst-dsexample
      • GstBaseTransfrom Class Functions
      • Other supporting functions
    • Enabling and configuring the sample plugin
    • Using the sample plugin in a custom application/pipeline
    • Implementing Custom Logic Within the Sample Plugin
    • Accessing NvBufSurface memory in OpenCV

TLT Integration with DeepStream

  • Transfer Learning Toolkit (TLT) Integration with DeepStream
    • Pre-trained models

Tutorials and How-to's

  • NvDCF parameter tuning guide
    • Accuracy-Performance Tradeoffs
    • Visual Feature Types and Feature Sizes
    • Robustness
      • Creation Policy
      • Termination Policy
    • Motion Dynamics
      • Moving-Average Estimator
      • Kalman Filter
    • Data Association
    • DCF Core Tuning
      • DCF Filter Learning
      • Instance-awareness
  • Custom YOLO Model in the DeepStream YOLO App
    • How to Use the Custom YOLO Model
      • Set up the sample

DeepStream Performance

  • Performance
    • TLT Pre-trained models
    • DeepStream reference model and tracker
    • DeepStream reference model
      • Data center GPU - T4
        • System Configuration
        • Application Configuration
      • Jetson
        • System Configuration
        • Jetson Nano
        • Jetson AGX Xavier
        • Jetson NX
        • Jetson TX2
        • Jetson TX1
    • Running applications using DLA

DeepStream Custom Model

  • Using a Custom Model with DeepStream
    • Custom Model Implementation Interface
    • Custom Output Parsing
    • IPlugin Implementation
      • How to Use IPluginCreator
      • How to Use IPluginFactory
        • For Caffe Files
        • For Uff Files
        • During Deserialization
    • Input Layer Initialization
    • CUDA Engine Creation for Custom Models
    • IModelParser Interface for Custom Model Parsing

DeepStream Key Features

  • Smart Video Record
  • IoT
    • Secure Edge-to-Cloud Messaging
      • 2-way TLS Authentication
        • Overview of Steps
        • TLS Version
        • Key generation
        • Certificate Signing
        • Choice of Cipher
        • Configure TLS options in Kafka config file for DeepStream
      • SASL/Plain
        • Overview of Steps
        • TLS Configuration
        • Credential Storage
        • Choosing Between 2-way TLS and SASL/Plain
      • Impact on performance
    • Bidirectional Messaging
      • Edge-to-Cloud
      • Cloud-to-Edge
        • NvMsgbroker Library
  • On the Fly Model Update
    • Assumptions
  • NTP Timestamp in DeepStream

DeepStream Application Migration

  • Application Migration to DeepStream 5.0 from DeepStream 4.X
    • Major Application Differences with DeepStream 4.X
    • Running DeepStream 4.x compiled Apps in DeepStream 5.0
    • Compiling DeepStream 4.X Apps in DeepStream 5.0

DeepStream Plugin Guide

  • GStreamer Plugin Overview
  • MetaData in the DeepStream SDK
    • NvDsBatchMeta: Basic Metadata Structure
    • User/Custom Metadata Addition inside NvDsBatchMeta
    • Adding Custom Meta in Gst Plugins Upstream from Gst-nvstreammux
      • Adding metadata to the plugin before Gst-nvstreammux
      • New metadata fields
  • Gst-nvinfer
    • Inputs and Outputs
    • Features
    • Gst-nvinfer File Configuration Specifications
    • Gst Properties
    • Tensor Metadata
      • To read or parse inference raw tensor data of output layers
    • Segmentation Metadata
  • Gst-nvinferserver
    • Inputs and Outputs
    • Gst-nvinferserver File Configuration Specifications
    • Features
    • Gst Properties
    • Triton Ensemble Models
    • Tensor Metadata
      • To read or parse inference raw tensor data of output layers
    • Segmentation Metadata
  • Gst-nvtracker
    • Inputs and Outputs
    • Gst Properties
    • Custom Low-Level Library
    • Low-Level Tracker Library Comparisons and Tradeoffs
    • NvDCF Low-Level Tracker
    • IOU Low-Level Tracker
    • KLT Low-Level Tracker
  • Gst-nvstreammux
    • Inputs and Outputs
    • Features
    • Gst Properties
  • Gst-nvstreamdemux
    • Inputs and Outputs
      • Use case 1
      • Use case 2
      • Use case 3
  • Gst-nvmultistreamtiler
    • Inputs and Outputs
    • Features
    • Gst Properties
  • Gst-nvdsosd
    • Inputs and Outputs
    • Features
    • Gst Properties
  • Gst-nvvideoconvert
    • Inputs and Outputs
    • Features
    • Gst Properties
  • Gst-nvdewarper
    • Inputs and Outputs
    • Features
    • Configuration File Parameters
    • Gst Properties
  • Gst-nvof
    • Inputs and Outputs
    • Features
    • Gst Properties
  • Gst-nvofvisual
    • Inputs and Outputs
    • Features
    • Gst Properties
  • Gst-nvsegvisual
    • Inputs and Outputs
    • Features
    • Gst Properties
  • Gst-nvvideo4linux2
    • Decoder
      • Inputs and Outputs
      • Features
      • Gst Properties
    • Encoder
      • Inputs and Outputs
      • Features
      • Gst Properties
  • Gst-nvjpegdec
    • Inputs and Outputs
    • Features
    • Gst Properties
  • Gst-nvmsgconv
    • Inputs and Outputs
    • Features
    • Gst Properties
    • Schema Customization
    • Payload with Custom Objects
  • Gst-nvmsgbroker
    • Inputs and Outputs
    • Features
    • Gst Properties
    • nvds_msgapi: Protocol Adapter Interface
      • nvds_msgapi_connect(): Create a Connection
      • nvds_msgapi_send() and nvds_msgapi_send_async(): Send an event
      • nvds_msgapi_subscribe(): Consume data by subscribing to topics
      • nvds_msgapi_do_work(): Incremental Execution of Adapter Logic
      • nvds_msgapi_disconnect(): Terminate a Connection
      • nvds_msgapi_getversion(): Get Version Number
      • nvds_msgapi_get_protocol_name(): Get name of the protocol
      • nvds_msgapi_connection_signature(): Get Connection signature
    • nvds_kafka_proto: Kafka Protocol Adapter
      • Installing Dependencies
      • Using the Adapter
      • Configuring Protocol Settings
      • Programmatic Integration
      • Security for Kafka
    • Azure MQTT Protocol Adapter Libraries
      • Installing Dependencies
      • Setting Up Azure IoT
      • Configuring Adapter Settings
      • Using the Adapter
        • Connection Details for the Device Client Adapter
        • Connection Details for the Module Client Adapter
      • Monitor Adapter Execution
        • Azure device client library log messages
        • Azure Module Client Library Log Messages
      • Message Topics and Routes
    • AMQP Protocol Adapter
      • Installing Dependencies
      • AMQP broker
      • Configure Adapter Settings
      • Using the Adapter
      • Programmatic Integration
      • Monitor Adapter Execution
    • nv_msgbroker: Message Broker interface
      • nv_msgbroker_connect(): Create a Connection
      • nv_msgbroker_send_async(): Send an event asynchronously
      • nv_msgbroker_subscribe(): Consume data by subscribing to topics
      • nv_msgbroker_disconnect(): Terminate a Connection
      • nv_msgbroker_version(): Get Version Number
      • nvds_logger: Logging Framework
        • Enabling Logging
        • Filtering Logs
        • Retiring and Managing Logs
        • Generating Logs
  • Gst-nvdsanalytics
    • Inputs and Outputs
    • Features
    • Gst Properties
    • Configuration File Parameters

DeepStream Troubleshooting and FAQ

  • Troubleshooting
    • You are migrating from DeepStream 4.0+ to DeepStream 5.0
    • “NvDsBatchMeta not found for input buffer” error while running DeepStream pipeline
    • The DeepStream reference application fails to launch, or any plugin fails to load
    • Application fails to run when the neural network is changed
    • The DeepStream application is running slowly (Jetson only)
    • The DeepStream application is running slowly
    • NVIDIA Jetson Nano™, deepstream-segmentation-test starts as expected, but crashes after a few minutes rebooting the system
    • Errors occur when deepstream-app is run with a number of streams greater than 100
    • Errors occur when deepstream-app fails to load plugin Gst-nvinferserver on dGPU only
    • Tensorflow models are running into OOM(Out-Of-Memory) problem
  • NvDCF tracker parameter tuning
    • Ghost bbox lingering
    • BBox flickering
    • Frequent tracking ID changes although no nearby objects
    • Frequent tracking ID switches to the nearby objects
  • Frequently Asked Questions
    • DeepStream General topics
      • How do I uninstall DeepStream?
      • What types of input streams does DeepStream 5.0 support?
      • What’s the throughput of H.264 and H.265 decode on dGPU (Tesla)?
      • How can I run the DeepStream sample application in debug mode?
      • Where can I find the DeepStream sample applications?
      • How can I verify that CUDA was installed correctly?
      • How can I interpret frames per second (FPS) display information on console?
      • My DeepStream performance is lower than expected. How can I determine the reason?
      • How can I specify RTSP streaming of DeepStream output?
      • What is the official DeepStream Docker image and where do I get it?
      • What is the recipe for creating my own Docker image?
      • How can I display graphical output remotely over VNC? How can I determine whether X11 is running?
      • Why does the deepstream-nvof-test application show the error message “Device Does NOT support Optical Flow Functionality” if run with NVIDIA Tesla P4 or NVIDIA Jetson Nano, Jetson TX2, or Jetson TX1?
      • Why is the Gst-nvstreammux plugin required in DeepStream 4.0+?
      • Why is a Gst-nvegltransform plugin required on a Jetson platform upstream from Gst-nveglglessink?
      • How do I profile DeepStream pipeline?
      • How can I check GPU and memory utilization on a dGPU system?
      • What is the approximate memory utilization for 1080p streams on dGPU?
      • What trackers are included in DeepStream and which one should I choose for my application?
      • When deepstream-app is run in loop on Jetson AGX Xavier using “while true; do deepstream-app -c <config_file>; done;”, after a few iterations I see low FPS for certain iterations. Why is that?
      • Why do I get the error Makefile:13: *** "CUDA_VER is not set".  Stop when I compile DeepStream sample applications?
      • How can I construct the DeepStream GStreamer pipeline?
      • Why am I getting “ImportError: No module named google.protobuf.internal when running convert_to_uff.py on Jetson AGX Xavier”?
      • Does DeepStream Support 10 Bit Video streams?
      • What is the difference between batch-size of nvstreammux and nvinfer? What are the recommended values for nvstreammux batch-size?
      • Why do some caffemodels fail to build after upgrading to DeepStream 5.0?
      • How do I configure the pipeline to get NTP timestamps?
      • Why is the NTP timestamp value 0?
      • How to handle operations not supported by Triton Inference Server?
      • Why do I see confidence value as -0.1.?
      • Why do I see tracker_confidence value as -0.1.?
      • Why do I see the below Error while processing H265 RTSP stream?
      • Why do I observe a lot of buffers being dropped When running deepstream-nvdsanalytics-test application on Jetson Nano ?
      • Why do I observe: A lot of buffers are being dropped. When running live camera streams even for few or single stream, also output looks jittery?
      • Why does the RTSP source used in gst-launch pipeline through uridecodebin show blank screen followed by the error - WARNING: from element /GstPipeline:pipeline0/GstNvStreamMux:m: No Sources found at the input of muxer. Waiting for sources?
      • What if I do not get expected 30 FPS from camera using v4l2src plugin in pipeline but instead get 15 FPS or less than 30 FPS?
      • On Jetson platform, I get same output when multiple Jpeg images are fed to nvv4l2decoder using multifilesrc plugin. Why is that?
      • How do I obtain individual sources after batched inferencing/processing? What are the sample pipelines for nvstreamdemux?
      • Why do I encounter such error while running Deepstream pipeline memory type configured and i/p buffer mismatch ip_surf 0 muxer 3?
      • How does secondary GIE crop and resize objects?
      • How to save frames from GstBuffer?
      • What are different Memory types supported on Jetson and dGPU?
      • What are different Memory transformations supported on Jetson and dGPU?
      • Why does my image look distorted if I wrap my cudaMalloc’ed memory into NvBufSurface and provide to NvBufSurfTransform?
      • Why am I getting following waring when running deepstream app for first time?
      • How to find out the maximum number of streams supported on given platform?
      • How to find the performance bottleneck in DeepStream?
    • Smart Record
      • Does smart record module work with local video streams?
      • Are multiple parallel records on same source supported?
      • What if I forgot to stop the recording?
      • I started the record with a set duration. Can I stop it before that duration ends?
      • What if I don’t set default duration for smart record?
      • What if I don’t set video cache size for smart record?
      • What is maximum duration of data I can cache as history for smart record?
      • Can I record the video with bounding boxes and other information overlaid?
    • Triton
      • Which Triton version is supported in DeepStream 5.0 GA release?
      • Can Jetson platform support the same features as dGPU for Triton plugin?
      • Can Gst-nvinfereserver (DeepSream Triton plugin) run on Nano platform?
      • How to enable TensorRT optimization for Tensorflow and ONNX models?
      • How to tune GPU memory for Tensorflow models?
      • Can Gst-nvinferserver support models cross processes or containers?
      • Can users set different model repos when running multiple Triton models in single process?
      • What is the difference between DeepStream classification and Triton classification?
      • Why is there a reshape in certain Triton config files?
      • How to support Triton ensemble model?
      • Does Gst-nvinferserver support Triton multiple instance groups?
      • Can Gst-nvinferserver support inference on multiple GPUs?
      • What is batch-size differences for a single model in different config files (gie group in source, config_inferserver.., and Triton model’s config.pbtxt)?

DeepStream API Guide

  • DeepStream API Guides

DeepStream Legal Information

  • DeepStream Legal Information
    • Notice
    • Trademarks
    • Copyright

DeepStream Feedback

  • Feedback form
    • Request Documentation Fix
DeepStream
  • Docs »
  • DeepStream Legal Information
  • View page source

DeepStream Legal Information¶

Notice¶

ALL NVIDIA DESIGN SPECIFICATIONS, REFERENCE BOARDS, FILES, DRAWINGS, DIAGNOSTICS, LISTS, AND OTHER DOCUMENTS (TOGETHER AND SEPARATELY, ”MATERIALS”) ARE BEING PROVIDED ”AS IS.” NVIDIA MAKES NO WARRANTIES, EXPRESS, IMPLIED, STATUTORY, OR OTHERWISE WITH RESPECT TO THE MATERIALS, AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OR CONDITION OF TITLE, MERCHANTABILITY, SATISFACTORY QUALITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT, ARE HEREBY EXCLUDED TO THE MAXIMUM EXTENT PERMITTED BY LAW.

Information furnished is believed to be accurate and reliable. However, NVIDIA Corporation assumes no responsibility for the consequences of use of such information or for any infringement of patents or other rights of third parties that may result from its use. No license is granted by implication or otherwise under any patent or patent rights of NVIDIA Corporation. Specifications mentioned in this publication are subject to change without notice. This publication supersedes and replaces all information previously supplied. NVIDIA Corporation products are not authorized for use as critical components in life support devices or systems without express written approval of NVIDIA Corporation.

Trademarks¶

NVIDIA, the NVIDIA logo, CUDA, Jetson, Jetson Nano, NVIDIA AGX, Tegra, TensorRT, Tesla and Xavier are trademarks or registered trademarks of NVIDIA Corporation in the United States and other countries. Other company and product names may be trademarks of the respective companies with which they are associated. The Android robot is reproduced or modified from work created and shared by Google and is used according to terms described in the Creative Commons 3.0 Attribution License. HDMI, the HDMI logo, and High-Definition Multimedia Interface are trademarks or registered trademarks of HDMI Licensing LLC. ARM, AMBA, and ARM Powered are registered trademarks of ARM Limited. Cortex, MPCore and Mali are trademarks of ARM Limited. All other brands or product names are the property of their respective holders. ”ARM” is used to represent ARM Holdings plc; its operating company ARM Limited; and the regional subsidiaries ARM Inc.; ARM KK; ARM Korea Limited.; ARM Taiwan Limited; ARM France SAS; ARM Consulting (Shanghai) Co. Ltd.; ARM Germany GmbH; ARM Embedded Technologies Pvt. Ltd.; ARM Norway, AS and ARM Sweden AB.

Copyright¶

© 2020 by NVIDIA Corporation. All rights reserved.

Next Previous

© Copyright 2019-2020, NVIDIA. Last updated on Oct 21, 2020.

Built with Sphinx using a theme provided by Read the Docs.