NVIDIA DeepStream SDK Developer Guide

DeepStream Getting Started

• Welcome to the DeepStream Documentation
  • NVIDIA DeepStream Overview
• Readme First
  • Additional Resources
• Quickstart Guide
  • Jetson Setup [ Not applicable for NVAIE customers ]
  • dGPU Setup for Ubuntu
  • dGPU Setup for RedHat Enterprise Linux (RHEL)
  • How to visualize the output if the display is not attached to the system
  • Platform and OS Compatibility
  • DeepStream Triton Inference Server Usage Guidelines
  • Using DLA for inference
• Docker Containers
  • Prerequisites

DeepStream Samples

• C/C++ Sample Apps Source Details
  • Plugin and Library Source Details
• Python Sample Apps and Bindings Source Details
  • Sample Application Source Details
  • Python Bindings and Application Development
  • MetaData Access
• DeepStream Reference Application - deepstream-app
  • Application Architecture
  • Reference Application Configuration
  • Configuration Groups
  • Application Tuning for DeepStream SDK
• DeepStream Reference Application - deepstream-test5 app
  • IoT Protocols supported and cloud configuration
  • Message consumer
  • Smart Record - Event based recording
  • OTA model update
  • Sensor Provisioning Support over REST API (Runtime sensor add/remove capability)
• DeepStream Reference Application - deepstream-audio app
  • DeepStream Audio Reference Application Architecture and Sample Graphs
• DeepStream Reference Application - deepstream-nmos app
  • Application Architecture
  • Using Easy-NMOS for NMOS Registry and Controller
• DeepStream Reference Application on GitHub
  • Use case applications
  • AI models with DeepStream
  • DeepStream features sample
• Sample Configurations and Streams
  • Contents of the package
• Implementing a Custom GStreamer Plugin with OpenCV Integration Example
  • Description of the Sample Plugin: gst-dsexample
  • Enabling and configuring the sample plugin
  • Using the sample plugin in a custom application/pipeline
  • Implementing Custom Logic Within the Sample Plugin
  • Adding NVTX APIs for sample plugin
  • Accessing NvBufSurface memory in OpenCV

TAO toolkit Integration with DeepStream

• TAO Toolkit Integration with DeepStream
  • Pre-trained models

Tutorials and How-to's

• Custom YOLO Model in the DeepStream YOLO App
  • How to Use the Custom YOLO Model
• NvMultiObjectTracker Parameter Tuning Guide
  • Accuracy-Performance Tradeoffs
  • Robustness
  • State Estimation
  • Data Association
  • DCF Core Tuning
• DeepStream 3D Custom Manual
  • DS3D Application Examples
  • ds3d::dataloader - Load Custom Lib for Data Capture
  • ds3d::datafilter- Loads Custom Lib for Input and Output Processing
  • ds3d::datarender - Loads Custom Lib for Data Rendering
  • DS3D GuardDataMap Buffer Management
  • Custom Libs Configuration Specifications

DeepStream Performance

• Performance
  • TAO Pre-trained models
  • DeepStream reference model and tracker
  • Configuration File Settings for Performance Measurement
  • DeepStream reference model

DeepStream Custom Model

• Using a Custom Model with DeepStream
  • Custom Model Implementation Interface
  • Custom Output Parsing
  • IPlugin Implementation
  • Input Layer Initialization
  • CUDA Engine Creation for Custom Models
  • IModelParser Interface for Custom Model Parsing

DeepStream Key Features

• Smart Video Record
  • Smart Video Record Module APIs
  • Smart Video Record Configurations
• IoT
  • Secure Edge-to-Cloud Messaging
  • Bidirectional Messaging
  • Autoreconnect feature
• On the Fly Model Update
  • Assumptions
• NTP Timestamp in DeepStream
• AV Sync in DeepStream
  • Setup for RTMP/RTSP Input streams for testing
  • AVSync Reference Pipelines
• DeepStream With REST API Sever (Alpha)
  • Features
  • nvds_rest_server
  • deepstream-server-app
  • Sensor provisioning with deepstream-test5-app
  • How to add custom REST API support
  • Limitations
• DeepStream 3D Action Recognition App
  • Getting Started
  • DeepStream 3D Action Recognition App Configuration Specifications
  • Build Custom sequence preprocess lib and application From Source
• DeepStream 3D Depth Camera App
  • Prerequisites
  • Depth Color Capture to 2D Rendering Pipeline Overview
  • Depth Color Capture to 3D Point Cloud Processing and Rendering
  • Getting Started
  • Run 3D Depth Capture, Point Cloud filter, and 3D Points Rendering Examples
  • DeepStream 3D Depth Camera App Configuration Specifications
• DeepStream Lidar Inference App (Alpha)
  • Prerequisites
  • Lidar Point Cloud to 3D Point Cloud Processing and Rendering
  • Getting Started
  • Run Lidar Point Cloud Data File reader, Point Cloud Inferencing filter, and Point Cloud 3D rendering and data dump Examples
  • DeepStream Lidar Inference App Configuration Specifications
• Networked Media Open Specifications (NMOS) in DeepStream
  • Supported Specifications
  • Prerequisites
  • NvDsNmos API
  • Sample Application
• Gst-nvdspostprocess in DeepStream
  • Detector models
  • Primary Classification model
  • Mask RCNN Model
  • Custom Parsing functions
• DeepStream Can Orientation App
  • Getting Started
  • DeepStream Can Orientation App Configuration Specifications
  • Build application From Source

DeepStream Application Migration

• Application Migration to DeepStream 6.2 from DeepStream 6.1
  • Running DeepStream 6.1 compiled Apps in DeepStream 6.2
  • Compiling DeepStream 6.1 Apps in DeepStream 6.2

Plugins Development Guide

DeepStream Plugin Guide

• GStreamer Plugin Overview
• MetaData in the DeepStream SDK
  • NvDsBatchMeta: Basic Metadata Structure
  • User/Custom Metadata Addition inside NvDsBatchMeta
  • Adding Custom Meta in Gst Plugins Upstream from Gst-nvstreammux
• Gst-nvdspreprocess (Alpha)
  • Inputs and Outputs
  • Features
  • Custom library Interfaces
  • Gst-nvdspreprocess File Configuration Specifications
  • Gst Properties
  • Sample pipelines
• Gst-nvinfer
  • Inputs and Outputs
  • Features
  • Gst-nvinfer File Configuration Specifications
  • Gst Properties
  • Clustering algorithms supported by nvinfer
  • Tensor Metadata
  • Segmentation Metadata
• Gst-nvinferaudio
  • Inputs and Outputs
  • Features
  • Gst Properties
• Gst-nvinferserver
  • Inputs and Outputs
  • Gst-nvinferserver Configuration File Specifications
  • Features
  • Gst Properties
  • DeepStream Triton samples
  • DeepStream Triton gRPC support
  • Triton Ensemble Models
  • Custom Process interface IInferCustomProcessor for Extra Input, LSTM Loop, Output Data Postprocess
  • Tensor Metadata Output for Downstream Plugins
  • Segmentation Metadata
• Gst-nvtracker
  • Inputs and Outputs
  • Gst Properties
  • NvDsTracker API for Low-Level Tracker Library
  • NvMultiObjectTracker : A Reference Low-Level Tracker Library
  • Unified Tracker Architecture for Composable Multi-Object Tracker
  • Workflow and Core Modules in The NvMultiObjectTracker Library
  • Low-Level Tracker Comparisons and Tradeoffs
  • IOU Tracker
  • NvSORT Tracker
  • NvDeepSORT Tracker
  • NvDCF Tracker
  • Setup and Visualization of Tracker Sample Pipelines
  • How to Implement a Custom Low-Level Tracker Library
• Gst-nvstreammux
  • Inputs and Outputs
  • Features
  • Gst Properties
• Gst-nvstreammux New
  • Inputs and Outputs
  • Features
  • Gst Properties
  • Mux Config Properties
  • NvStreamMux Tuning Solutions for specific use cases
  • Known Issues and FAQ
• Gst-nvstreamdemux
  • Inputs and Outputs
• Gst-nvmultistreamtiler
  • Inputs and Outputs
  • Features
  • Gst Properties
• Gst-nvdsosd
  • Inputs and Outputs
  • Features
  • Gst Properties
• Gst-nvdsmetautils
  • Gst-nvdsmetainsert
  • Inputs and Outputs
  • Features
  • customlib_impl Interfaces
  • Gst Properties
  • Use Cases & Sample Pipelines
  • Gst-nvdsmetaextract
  • Inputs and Outputs
  • Features
  • customlib_impl Interfaces
  • Gst Properties
  • Use Cases & Sample Pipelines
• Gst-nvdsvideotemplate
  • Inputs and Outputs
  • Features
  • customlib_impl Interfaces
  • Gst Properties
  • Sample pipelines
• Gst-nvdsaudiotemplate
  • Inputs and Outputs
  • Features
  • customlib_impl Interfaces
  • Gst Properties
  • Sample pipelines
• Gst-nvvideoconvert
  • Inputs and Outputs
  • Features
  • Gst Properties
• Gst-nvdewarper
  • Inputs and Outputs
  • Features
  • Configuration File Parameters
  • Gst Properties
• Gst-nvof
  • Inputs and Outputs
  • Features
  • Gst Properties
• Gst-nvofvisual
  • Inputs and Outputs
  • Features
  • Gst Properties
• Gst-nvsegvisual
  • Inputs and Outputs
  • Features
  • Gst Properties
• Gst-nvvideo4linux2
  • Decoder
  • Encoder
• Gst-nvjpegdec
  • Inputs and Outputs
  • Features
  • Gst Properties
• Gst-nvmsgconv
  • Inputs and Outputs
  • Features
  • Gst Properties
  • NvMsgConv low level library API’s
  • Schema Customization
  • Payload with Custom Objects
• Gst-nvmsgbroker
  • Inputs and Outputs
  • Features
  • Gst Properties
  • nvds_msgapi: Protocol Adapter Interface
  • nvds_kafka_proto: Kafka Protocol Adapter
  • Azure MQTT Protocol Adapter Libraries
  • AMQP Protocol Adapter
  • REDIS Protocol Adapter
  • nv_msgbroker: Message Broker interface
• Gst-nvdsanalytics
  • Inputs and Outputs
  • Features
  • Gst Properties
  • Configuration File Parameters
• Gst-nvdsasr
  • Inputs and Outputs
  • Features
  • DS-Riva ASR Library YAML File Configuration Specifications
  • Gst Properties
• Gst-nvds_text_to_speech (Alpha)
  • Inputs and Outputs
  • Features
  • DS-Riva TTS Yaml File Configuration Specifications
  • Gst Properties
  • Sample Application
• Gst-nvdsudpsrc
  • Inputs and Outputs
  • Features
  • Gst Properties
  • Example pipelines
• Gst-nvdsudpsink
  • Inputs and Outputs
  • Features
  • Gst Properties
  • Example pipelines
• Gst-nvdspostprocess (Alpha)
  • Inputs and Outputs
  • Features
  • Gst-nvdspostprocess File Configuration Specifications
  • Gst Properties
  • Sample pipelines
• Gst-nvds3dfilter
  • Inputs and Outputs
  • Custom library Interfaces
  • Gst-nvds3dfilter properties Specifications
• Gst-nvdsucx (Alpha)
  • Description
  • Requirements
  • Inputs and Outputs
  • Gst Properties
  • Examples
• Gst-nvdsxfer
  • Inputs and Outputs
  • Gst Properties
  • How to test
  • Use cases
• Gst-nvvideotestsrc
  • Outputs
  • Gst Properties
  • How to test
• Gst-nvmultiurisrcbin (Alpha)
  • 1. Introduction
  • 2. Features
  • 3. How to use nvmultiurisrcbin in a pipeline
  • 4. Gst Properties
  • 5. Important Notes
  • 6. Known Issues
• Gst-nvurisrcbin
  • 1. Introduction
  • 2. How to use nvurisrcbin in a pipeline
  • 3. Gst Properties

Troubleshooting and FAQ Guide

DeepStream Troubleshooting and FAQ

• Troubleshooting
  • You are migrating from DeepStream 6.0 to DeepStream 6.2
  • Inference
  • Performance
  • Triton
  • Troubleshooting in Tracker Setup and Parameter Tuning
  • Graph Composer Troubleshooting
  • Miscellaneous
• Frequently Asked Questions
  • DeepStream General topics
  • Smart Record
  • Triton

DeepStream API Reference Documentation

DeepStream API Guide

• DeepStream API Guides

DeepStream Legal Information

• DeepStream Legal Information
  • Notice
  • Trademarks
  • Copyright

DeepStream Feedback

• Feedback form
  • Request Documentation Fix