Gst-nvof
============

NVIDIA GPUs, starting with the dGPU Turing generation (except TU117 GPU) and Jetson Xavier generation, contain a hardware accelerator for computing optical flow. Optical flow vectors are useful in various use cases such as object detection and tracking, video frame rate up-conversion, depth estimation, stitching, and so on.
The Gst-nvof plugin collects a pair of NV12 images and passes it to the low-level optical flow library. The low-level library returns a map of flow vectors between the two frames as its output.
The map of flow vectors is encapsulated in the ``NvDsOpticalFlowMeta`` structure and is added as a user meta with meta_type set to ``NVDS_OPTICAL_FLOW_META``. The user meta is added to the frame_user_meta_list member of NvDsFrameMeta.
For guidance on how to access user metadata, see User/Custom Metadata Addition Inside ``NvDsMatchMeta`` and Tensor Metadata.


.. image:: /content/DS_plugin_gst-nvof.png
         :align: center
         :alt: Gst-Nvof

.. note::
   Flow vectors are bi-dimensional vector and each component of flow vector is stored using S10.5 in a fixed-point representation of a 16-bit number. Structure ``NvOFFlowVector`` is used to store Flow Vector. ::

       /**
       * Holds motion vector information about an element.
       */
       typedef struct _NvOFFlowVector
       {
       /** Holds the motion vector X component. */
       gshort flowx;

       /** Holds the motion vector Y component. */
       gshort flowy;
       } NvOFFlowVector;

   MSB specifies the sign bit, the next 10 bits specify the integer value and the last 5 bits represent the fractional value of the flow vector. HW support only quarter pixel resolution and only the first 2 bits from 5 available fraction bits are used. We can convert the flow vector from fixed-point representation to floating-point using the following formula. ::

        NvOFFlowVector   output;
        float32_t flow_vector.x = output.flowx/32.0f;
        float32_t flow_vector.y = output.flowy/32.0f;                  

Inputs and Outputs
-------------------

* Inputs

  *	GStreamer buffer containing NV12 frame(s)

* Control parameters

  *	gpu-id: selects the GPU ID (valid only for dGPU platforms)
  *	dump-of-meta: enables dumping of optical flow map vector into a .bin file
  *	preset-level: sets the preset level
  *	pool-size: sets the pool size
  *	grid-size: sets the grid size

* Outputs

  *	GStreamer buffer containing NV12 frame(s)
  *	NvDsOpticalFlowMeta metadata

Features
---------

The following table summarizes the features of the plugin.

.. csv-table:: Gst-nvof plugin features 
  :file: ../text/tables/Gst-Nvof tables/DS_Plugin_gst-nvof_features.csv
  :widths: 30, 30, 30
  :header-rows: 1


Gst Properties
----------------

The following table describes the Gst properties of the Gst-nvof plugin.

.. csv-table:: Gst-nvof plugin features 
  :file: ../text/tables/Gst-Nvof tables/DS_Plugin_gst-nvof_gst_properties.csv
  :widths: 20, 20, 20, 20
  :header-rows: 1