Gst-nvdewarper =============== This plugin dewarps camera input. It accepts gpu-id and config-file as properties. Based on the selected configuration of surfaces, it can generate a maximum of four dewarped surfaces. It currently supports 18 dewarping projection types. Out of these, ``NVDS_META_SURFACE_FISH_PUSHBROOM`` and ``NVDS_META_SURFACE_FISH_VERTCYL`` are used in 360-D use case for dewarping 360° camera input. The plugin performs its function in these steps: 1. Reads the configuration file and creates a vector of surface configurations. It supports a maximum of four dewarp surface configurations. 2. Receives the frame from the decoder. Based on the configuration, generates up to four dewarped surfaces. 3. Scales these surfaces down to network/selected dewarper output resolution using `NPP` APIs. 4. Pushes a buffer containing the dewarped surfaces to the downstream component. .. image:: /content/DS_plugin_gst-nvdewarper.png :align: center :alt: Gst-nvdewarper Inputs and Outputs ------------------- * Inputs * A buffer containing a frame in RGBA format * Control parameters * gpu-id; selects the GPU ID (dGPU only) * config-file, containing the pathname of the dewarper configuration file * Output * Dewarped RGBA surfaces * ``NvDewarperSurfaceMeta`` with information associated with each surface (`projection_type`, `surface_index`, and `source_id`), and the number of valid dewarped surfaces in the buffer (`num_filled_surfaces`) Features --------- The following table summarizes the features of the plugin. .. csv-table:: Gst-nvdewarper features :file: ../text/tables/Gst-nvdewarper tables/DS_Plugin_gst-nvdewarper_features.csv :widths: 30, 30, 30 :header-rows: 1 Configuration File Parameters ------------------------------ The configuration file located at `configs/deepstream-app/config_dewarper.txt`. The configuration file specifies per-surface configuration parameters in ``[surface]`` groups, where `` is an integer from `0` to `3`, representing dewarped surfaces `0` to `3`. .. csv-table:: Gst-nvdewarper configuration file [surface] parameters :file: ../text/tables/Gst-nvdewarper tables/DS_Plugin_gst-nvdewarper_config_surface_n_parameters.csv :widths: 20, 20, 20, 20, 20 :header-rows: 1 .. note:: In case viewing angles (top & bottom)/dst-focal-length/dst-principal-point are not provided, the destination center of projection (Principal Point) and Focal Length are kept the same as the source. To increase the flexibility, `num-batch-buffers` is also a `GstProperty`. The plugin properties are parsed in order, so if `num-batch-buffers` is passed both attached to the `GstElement` in the `gstreamer` pipeline (eg. with ``gst-launch-1.0`` command), and in the configuration file, the latest configuration applied will be used (see example below). *Example:* * Config file (``config_dewarper.txt``) with ``num-batch-buffers=2``. * Command Line argument with property ``num-batch-buffers=1``. *Pipelines:* * ``gst-launch-1.0 … ! nvdewarper config-file=config_dewarper.txt num-batch-buffers=1 ! …`` * ``gst-launch-1.0 … ! nvdewarper num-batch-buffers=1 config-file=config_dewarper.txt ! …`` Pipeline 1 will result in 1 surface per buffer whereas pipeline 2 will result in 2 surfaces per buffer. This plugin can be tested with the one of the following pipelines. * For dGPU: :: gst-launch-1.0 filesrc location=streams/sample_cam6.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! nvvideoconvert ! nvdewarper config-file=config_dewarper.txt source-id=6 nvbuf-memory-type=3 ! m.sink_0 nvstreammux name=m width=960 height=752 \ batch-size=4 num-surfaces-per-frame=4 ! nvmultistreamtiler ! nveglglessink Perspective Projection use-case: :: gst-launch-1.0 filesrc location=streams/yoga.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! nvvideoconvert ! nvdewarper config-file=config_dewarper_perspective.txt nvbuf-memory-type=3 ! m.sink_0 nvstreammux name=m width=3680 height=2428 \ batch-size=1 num-surfaces-per-frame=1 ! nvmultistreamtiler ! nveglglessink * For Jetson: :: gst-launch-1.0 filesrc location= streams/sample_cam6.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! nvvideoconvert ! nvdewarper config-file=config_dewarper.txt source-id=6 ! m.sink_0 nvstreammux name=m width=960 height=752 batch-size=4 \ num-surfaces-per-frame=4 ! nvmultistreamtiler ! nv3dsink Perspective Projection use-case: :: gst-launch-1.0 filesrc location=streams/yoga.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! nvvideoconvert ! nvdewarper config-file=config_dewarper_perspective.txt nvbuf-memory-type=3 ! m.sink_0 nvstreammux name=m width=3680 height=2428 \ batch-size=1 num-surfaces-per-frame=1 ! nvmultistreamtiler ! nv3dsink * Image File example dGPU: :: gst-launch-1.0 filesrc location=streams/yoga.jpg ! nvjpegdec ! nvdewarper config-file=config_dewarper_perspective.txt ! nvvideoconvert ! jpegenc ! filesink location=output.jpg * Image File example Jetson: :: gst-launch-1.0 filesrc location=streams/yoga.jpg ! nvjpegdec ! nvvideoconvert ! nvdewarper config-file=config_dewarper_perspective.txt ! nvvideoconvert ! jpegenc ! filesink location=output.jpg To better see the effect of cuda-address-mode you can change the dewarping size. For example, in config_dewarper_perspective.txt change [surface0] parameters to: :: width=7360 height=4856 cuda-address-mode=1 The Gst-nvdewarper plugin always outputs a `GStreamer` buffer which contains the number of dewarped surfaces defined by `num-batch-buffers` (currently maximum of four surfaces are supported). These dewarped surfaces are scaled to the output resolution (``output-width × output-height``) set in the configuration file. Also, the `num-surfaces-per-frame` and `batch-size` of `Gst-nvstreammux` must be set accordingly. Batch-size must be a multiple of the number of dewarped surfaces, which should be set in `num-surfaces-per-frame`. Gst Properties --------------- The following table describes the Gst-nvdewarper plugin’s Gst properties. .. csv-table:: Gst-nvdewarper gst properties :file: ../text/tables/Gst-nvdewarper tables/DS_Plugin_gst-nvdewarper_gst_properties.csv :widths: 25, 25, 25, 25 :header-rows: 1