The DeepStream SDK supports a mechanism to add third party or custom algorithms within the reference application by modifying the example plugin (gst-dsexample). The sources for the plugin are in sources/gst-plugins/gst-dsexample directory in the SDK.
This plugin was written for GStreamer 1.14.1 but is compatible with newer versions of GStreamer. This plugin derives from the GstBaseTransform class:
The GStreamer example plugin (gst-dsexample) demonstrates the following:
• Processing the entire frame, with downscaling / color conversion if required.
• Processing objects detected by the Primary Detector, specifically, cropping these objects from the frame and then processing the crops.
This release includes a simple static library dsexample_lib that demonstrates the interface between custom libraries and this Gstreamer plugin. The library generates simple labels of the form "Obj_label". The library implements these functions:
• DsExampleCtxInit—Initalizes the custom library
• DsExampleCtxDeinit—De-initalizes the custom library
• DsExampleProcess – Process on an input frame
The GStreamer plugin itself is a standard in-place transform plugin. Because it does not generate new buffers but only adds / updates existing metadata, the plugin implements an in-place transform. Some of the code is standard GStreamer plugin boilerplate (e.g. plugin_init, class_init, instance_init). Other functions of interest are as follows:
GstBaseTransfrom Class Functions
• start—Acquires resources, allocate memory, initialize example library.
• stop—De-initializes the example library and frees up resources and memory.
• set_caps—Gets the capabilities of the video (i.e. resolution, color format, framerate) that flow through this element. Allocations / initializations that depend on input video format can be done here.
• transform_ip—Finds the metadata of the primary detector.
• Call this function when the upstream element pushes a buffer.
• Use get_converted_mat to get the required buffer for pushing to library. Push the data to the example library. Pop the example library output.
• Attach / update metadata using attach_metadata_full_frame or attach_metadata_object.
Other supporting functions
• get_converted_mat—Scales, converts, or crops the input buffer, either the full frame or the object based on its co-ordinates in primary detector metadata.
• attach_metadata_full_frame—Shows how the plugin can attach its own metadata for objects detected by the plugin.
• attach_metadata_object—Shows how the plugin can update labels for objects detected by primary detector.
Note:
On Jetson devices, custom GStreamer plugins must export the environment variable DS_NEW_BUFAPI and set its value to 1. See gst_dsexample_class_init() for an example in a plugin (Gst-dsexample).
Enabling and configuring the sample plugin
The pre-compiled deepstream-app binary already has the functionality to parse the configuration and add the sample element to the pipeline.
To enable and configure the plugin, add the following section to an existing configuration file (for example, source4_720p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt):
[ds-example]
enable=1
processing-width=640
processing-height=480
full-frame=0
unique-id=15
Using the sample plugin in a custom application/pipeline
The sample plugin can be used in a gst-launch pipeline. The pipeline can also be constructed in a custom application.
To construct a pipeline for running the plugin in full frame mode
• Construct a pipeline for running the plugin in full frame mode with the following command.
