Gst-nvdspostprocess (Alpha) =========================== The `Gst-nvdspostprocess` plugin is a customizable plugin which provides a custom library interface for post processing on Tensor output of inference plugins (`nvinfer/nvinferserver`). Currently the plugin supports detection, classification and segmentation models for parsing. The plugin connects to post processing lib provided as plugin parameter. The parsing parameters can be specified via a config file. Refer to section :ref:`Gst-nvdspostprocess File Configuration Specifications` below for more details. For the plugin to perform parsing of inference tensor output, it is necessary to attach tensor output by setting ``output-tensor-meta`` as ``1`` in inference configuration file and disabling parsing in the inference plugin by setting ``network-type`` as ``100`` i.e. network type other. Similarly, in `Gst-nvinferserver` we can enable the following configs in the configuration file. ``output_control { output_tensor_meta : true }`` and to disable native post processing, update: ``infer_config { postprocess { other {} } }``. Default plugin implementation provides following functionalities. 1. It parses the tensor meta attached to frame meta or object meta. 2. Attaches the output of tensor parsing either to frame meta or object meta. The default custom library (postprocesslib_impl) provided with the plugin implements these functionalities. It can be configured to parse detection, classification and segmentation networks. For multiple inference plugins in pipeline require multiple post process plugin instances corresponding to them. .. image:: /content/DS_plugin_gst-nvdspostprocess.png :align: center :alt: Gst-nvdspostprocess Inputs and Outputs ------------------- * Inputs * Input Video Gst Buffers * Metadata (NvDsBatchMeta) * Tensor Meta (NvDsInferTensorMeta) * Control parameters * postprocesslib-config-file * postprocesslib-name * gpu-id * Output * Output Video Gst Buffers * Original Metadata (NvDsBatchMeta) (with addition of Object Meta/Classifier Meta/Segmentation Meta) Features --------- The following table summarizes the features of the plugin. .. csv-table:: Gst-nvdspostprocess features :file: ../text/tables/Gst-nvdspostprocess tables/DS_Plugin_gst-nvdspostprocess_features.csv :widths: 30, 30, 30 :header-rows: 1 .. _Gst-nvdspostprocess File Configuration Specifications: Gst-nvdspostprocess File Configuration Specifications ----------------------------------------------------- The Gst-nvdspreprocess configuration file uses a “YAML” format Refer to examples like ``config_detector.yml, config_classifier_car_color.yml, config_segmentation_semantic.yml`` located at ``/opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspostprocess/``. The ``property`` group configures the general behavior of the plugin. The ``class-attrs-all`` group configures parameters for all classes for detector post processor. The ``class-attrs-`` group configures parameters ```` specific class for detector post processor. The following two tables describes the keys supported for ``property`` groups and ``class-attrs-`` groups respectively. .. csv-table:: Gst-nvdspostprocess property Group Supported Keys :file: ../text/tables/Gst-nvdspostprocess tables/DS_Plugin_gst-nvdspostprocess_property_grp_keys.csv :widths: 20, 20, 20, 20 :header-rows: 1 .. csv-table:: Gst-nvdspostprocess class-attr- Group Supported Keys :file: ../text/tables/Gst-nvdspostprocess tables/DS_Plugin_gst-nvdspostprocess_class_attr_grp_keys.csv :widths: 20, 20, 20, 20 :header-rows: 1 Gst Properties -------------- The following table describes the Gst properties of the Gst-nvdspostprocess plugin. .. csv-table:: Gst-nvspostprocess gst properties :file: ../text/tables/Gst-nvdspostprocess tables/DS_Plugin_gst-nvdspostprocess_gst_properties.csv :widths: 20, 20, 20, 20 :header-rows: 1 Sample pipelines ---------------- Given below are some sample pipelines, please set appropriate configuration file and library paths. **For multi-stream detector and classifier (dGPU):** :: gst-launch-1.0 uridecodebin \ uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 \ ! m.sink_0 nvstreammux name=m width=1920 height=1080 batch-size=2 ! \ nvinfer config-file-path= \ /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt \ ! nvinfer config-file-path= config_infer_secondary_carcolor_postprocess.txt \ ! nvdspostprocess \ postprocesslib-config-file=config_classifier_car_color.yml \ postprocesslib-name= ./postprocesslib_impl/libpostprocess_impl.so ! nvinfer \ config-file-path= config_infer_secondary_carmake_postprocess.txt \ ! nvdspostprocess \ postprocesslib-config-file=config_classifier_car_make.yml \ postprocesslib-name= ./postprocesslib_impl/libpostprocess_impl.so ! nvinfer \ config-file-path= config_infer_secondary_vehicletypes_postprocess.txt \ ! nvdspostprocess postprocesslib-config-file= \ config_classifier_vehicle_type.yml postprocesslib-name= \ ./postprocesslib_impl/libpostprocess_impl.so ! nvmultistreamtiler ! nvvideoconvert \ gpu-id=0 ! nvdsosd ! nveglglessink sync=1 -v uridecodebin \ uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! m.sink_1 **For multi-stream detector and classifier (Jetson):** :: gst-launch-1.0 uridecodebin \ uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 \ ! m.sink_0 nvstreammux name=m width=1920 height=1080 batch-size=2 ! \ nvinfer config-file-path= \ /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt \ ! nvinfer config-file-path= config_infer_secondary_carcolor_postprocess.txt \ ! nvdspostprocess \ postprocesslib-config-file=config_classifier_car_color.yml \ postprocesslib-name= ./postprocesslib_impl/libpostprocess_impl.so ! nvinfer \ config-file-path= config_infer_secondary_carmake_postprocess.txt \ ! nvdspostprocess \ postprocesslib-config-file=config_classifier_car_make.yml \ postprocesslib-name= ./postprocesslib_impl/libpostprocess_impl.so ! nvinfer \ config-file-path= config_infer_secondary_vehicletypes_postprocess.txt \ ! nvdspostprocess postprocesslib-config-file= \ config_classifier_vehicle_type.yml postprocesslib-name= \ ./postprocesslib_impl/libpostprocess_impl.so ! nvmultistreamtiler ! nvvideoconvert \ gpu-id=0 ! nvdsosd ! nv3dsink sync=1 -v uridecodebin \ uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! m.sink_1 **For single stream segmentation model (dGPU):** :: gst-launch-1.0 uridecodebin \ uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h265.mp4 \ ! m.sink_0 nvstreammux name=m width=1920 height=1080 batch-size=1 ! \ nvinfer config-file-path= \ config_infer_segmentation_semantic_post_process.txt ! nvdspostprocess \ postprocesslib-config-file= config_segmentation_semantic.yml \ postprocesslib-name= ./postprocesslib_impl/libpostprocess_impl.so ! queue ! nvsegvisual ! \ nvmultistreamtiler ! nvvideoconvert gpu-id=0 ! nvdsosd ! nveglglessink \ sync=1 -v **For single stream segmentation model (Jetson):** :: gst-launch-1.0 uridecodebin \ uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h265.mp4 \ ! m.sink_0 nvstreammux name=m width=1920 height=1080 batch-size=1 ! \ nvinfer config-file-path= \ config_infer_segmentation_semantic_post_process.txt ! nvdspostprocess \ postprocesslib-config-file= config_segmentation_semantic.yml \ postprocesslib-name= ./postprocesslib_impl/libpostprocess_impl.so ! queue ! nvsegvisual ! \ nvmultistreamtiler ! nvvideoconvert gpu-id=0 ! nvdsosd ! nv3dsink \ sync=1 -v