NVIDIA Tegra
NVIDIA DeepStream SDK
4.0.2 Release

 
Implementing a Custom GStreamer Plugin
 
Description of the Sample Plugin: gst-dsexample
Enabling and configuring the sample plugin
Using the sample plugin in a custom application/pipeline
Implementing Custom Logic Within the Sample Plugin
The DeepStream SDK supports a mechanism to add third party or custom algorithms within the reference application by modifying the example plugin (gst-dsexample). The sources for the plugin are in sources/gst-plugins/gst-dsexample directory in the SDK.
This plugin was written for GStreamer 1.14.1 but is compatible with newer versions of GStreamer. This plugin derives from the GstBaseTransform class:
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-libs/html/GstBaseTransform.html
Description of the Sample Plugin: gst-dsexample
The GStreamer example plugin (gst-dsexample) demonstrates the following:
Processing the entire frame, with downscaling / color conversion if required.
Processing objects detected by the Primary Detector, specifically, cropping these objects from the frame and then processing the crops.
This release includes a simple static library dsexample_lib that demonstrates the interface between custom libraries and this Gstreamer plugin. The library generates simple labels of the form "Obj_label". The library implements these functions:
DsExampleCtxInit—Initalizes the custom library
DsExampleCtxDeinit—De-initalizes the custom library
DsExampleProcess – Process on an input frame
The GStreamer plugin itself is a standard in-place transform plugin. Because it does not generate new buffers but only adds / updates existing metadata, the plugin implements an in-place transform. Some of the code is standard GStreamer plugin boilerplate (e.g. plugin_init, class_init, instance_init). Other functions of interest are as follows:
GstBaseTransfrom Class Functions
start—Acquires resources, allocate memory, initialize example library.
stop—De-initializes the example library and frees up resources and memory.
set_caps—Gets the capabilities of the video (i.e. resolution, color format, framerate) that flow through this element. Allocations / initializations that depend on input video format can be done here.
transform_ip—Finds the metadata of the primary detector.
Call this function when the upstream element pushes a buffer.
Use get_converted_mat to get the required buffer for pushing to library. Push the data to the example library. Pop the example library output.
Attach / update metadata using attach_metadata_full_frame or attach_metadata_object.
Other supporting functions
get_converted_mat—Scales, converts, or crops the input buffer, either the full frame or the object based on its co-ordinates in primary detector metadata.
attach_metadata_full_frame—Shows how the plugin can attach its own metadata for objects detected by the plugin.
attach_metadata_object—Shows how the plugin can update labels for objects detected by primary detector.
Note:
On Jetson devices, custom GStreamer plugins must export the environment variable DS_NEW_BUFAPI and set its value to 1. See gst_dsexample_class_init() for an example in a plugin (Gst-dsexample).
Enabling and configuring the sample plugin
The pre-compiled deepstream-app binary already has the functionality to parse the configuration and add the sample element to the pipeline.
To enable and configure the plugin, add the following section to an existing configuration file (for example, source4_720p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt):
[ds-example]
enable=1
processing-width=640
processing-height=480
full-frame=0
unique-id=15
Using the sample plugin in a custom application/pipeline
The sample plugin can be used in a gst-launch pipeline. The pipeline can also be constructed in a custom application.
To construct a pipeline for running the plugin in full frame mode
Construct a pipeline for running the plugin in full frame mode with the following command.
For Jetson:
$ gst-launch-1.0 filesrc location= <mp4-file> ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1280 height=720 ! nvvideoconvert ! dsexample full-frame=1 <other-properties> ! nvdsosd ! nvegltransform ! nveglglessink
For Tesla:
$ gst-launch-1.0 filesrc location= <mp4-file> ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1280 height=720 ! nvvideoconvert ! dsexample full-frame=1 <other-properties> ! nvdsosd ! nveglglessink
To construct a pipeline for running the plugin to process on objects detected by the primary model
Construct a pipeline for running the plugin to process on objects detected by the primary model with the following command.
For Jetson:
$ gst-launch-1.0 filesrc location= <mp4-file> ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1280 height=720 ! nvinfer config-file-path= <primary-detector-config> ! nvvideoconvert ! dsexample full-frame=0 <other-properties> ! nvdsosd ! nvegltransform ! nveglglessink
For Tesla:
$ gst-launch-1.0 filesrc location= <mp4-file> ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1280 height=720 ! nvinfer config-file-path= <primary-detector-config> ! nvvideoconvert ! dsexample full-frame=0 <other-properties> ! nvdsosd ! nveglglessink
Implementing Custom Logic Within the Sample Plugin
To implement custom logic within the plugin, replace the following function calls list below with corresponding functions of any other custom library.
DsExampleCtxInit
DsExampleCtxDeinit
DsExampleProcess
Depending on the input requirements of the library, get_converted_mat may also require modification.