Gst-nvinferaudio
================
The Gst-nvinferaudio plugin does inferencing on input data using NVIDIA® TensorRT™.
The plugin accepts batched audio buffers from upstream. The NvDsBatchMeta structure must already be attached to the Gst Buffers.
The low-level library (libnvds_infer) operates on transformed audio data with dimension CHW.
The Gst-nvinferaudio plugin performs transform (log mel spectogram), on the input frame based on ``audio-transform`` property setting and transformed audio data is passed to the TensorRT engine for inferencing.
The output type generated by the low-level library depends on the network type.

Gst-nvinferaudio currently works on the following type of networks:

* Encoder Decoder RNN Architecture
* CNN

Detailed documentation of the TensorRT interface is available at:
https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html

  .. image:: /content/DS_plugin_gst-nvinferaudio.png
         :align: center
         :alt: Gst-nvinferaudio

Downstream components receive a Gst Buffer with unmodified contents plus the metadata created from the inference output of the Gst-nvinferaudio plugin.

Inputs and Outputs
~~~~~~~~~~~~~~~~~~~~

This section summarizes the inputs, outputs, and communication facilities of the Gst-nvinferaudio plugin.

* Inputs

  * Gst Buffer
  * NvDsBatchMeta (attaching NvDsFrameMeta)
  * ONNX
  * Layers: Supports all layers supported by TensorRT, see: https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html

|

* Control parameters
 
 Gst-nvinferaudio gets control parameters from a configuration file. You can specify this by setting the property config-file-path. For details, see Gst-nvinferaudio File Configuration Specifications. Other control parameters that can be set through GObject properties are:

  * Batch size
  * Inference interval
  * Attach inference tensor outputs as buffer metadata
  * Attach instance mask output as in object metadata
  * The parameters set through the GObject properties override the parameters in the Gst-nvinferaudio configuration file.


* Outputs

  * Gst Buffer
  * NvDsFrameMeta
  * NvDsClassifierMeta

Features
~~~~~~~~~
The following table summarizes the features of the plugin.
 
 .. csv-table:: Gst-nvinferaudio plugin features 
     :file: ../text/tables/Gst-nvinferaudio tables/DS_Plugin_gst-nvinferaudio_features.csv
     :widths: 30, 30, 30
     :header-rows: 1


The Gst-nvinferaudio configuration file uses a “Key File” format described in https://specifications.freedesktop.org/desktop-entry-spec/latest.
The [property] group configures the general behavior of the plugin. It is the only mandatory group.
The [class-attrs-all] group configures detection parameters for all classes.
The [class-attrs-<class-id>] group configures detection parameters for a class specified by <class-id>. For example, the [class-attrs-23] group configures detection parameters for class ID 23. This type of group has the same keys as [class-attrs-all].
The following two tables respectively describe the keys supported for [property] groups and [class-attrs-…] groups.

  .. csv-table:: Gst-nvinferaudio Property Group Supported Keys
     :file: ../text/tables/Gst-nvinferaudio tables/DS_Plugin_gst-nvinferaudio_propertygrp_keys.csv
     :widths: 20, 20, 20, 20, 20
     :header-rows: 1

  .. csv-table:: Gst-nvinferaudio Class-attributes Group Supported Keys
     :file: ../text/tables/Gst-nvinferaudio tables/DS_Plugin_gst-nvinferaudio_class_attr_grp_keys.csv
     :widths: 20, 20, 20, 20, 20
     :header-rows: 1

Gst Properties
~~~~~~~~~~~~~~~~~
The values set through Gst properties override the values of properties in the configuration file. The application does this for certain properties that it needs to set programmatically.
The following table describes the Gst-nvinferaudio plugin’s Gst properties.

.. csv-table:: Gst-nvinferaudio Gst Properties
     :file: ../text/tables/Gst-nvinferaudio tables/DS_Plugin_gst-nvinferaudio_gst_properties.csv
     :widths: 25, 25, 25, 25
     :header-rows: 1

.. csv-table:: Gst-nvinferaudio audio-transform parameters structure
     :file: ../text/tables/Gst-nvinferaudio tables/DS_Plugin_gst-nvinferaudio_audio_transform_params.csv
     :widths: 25, 25
     :header-rows: 1