Gst-nvdewarper
This plugin dewarps 360° camera input. It accepts gpu-id and config-file as properties. Based on the selected configuration of surfaces, it can generate a maximum of four dewarped surfaces. It currently supports dewarping of two projection types, NVDS_META_SURFACE_FISH_PUSHBROOM and NVDS_META_SURFACE_FISH_VERTCYL. Both are used in 360‑D use case.
The plugin performs its function in these steps:
1. Reads the configuration file and creates a vector of surface configurations. It supports a maximum of four dewarp surface configurations.
2. Receives the 360‑D frame from the decoder; based on the configuration, generates up to four dewarped surfaces.
3. Scales these surfaces down to network/ selected dewarper output resolution using NPP APIs.
4. Pushes a buffer containing the dewarped surfaces to the downstream component.
Inputs and Outputs
• Inputs
• A buffer containing a 360‑D frame in RGBA format
• Control parameters
• gpu-id; selects the GPU ID (dGPU only)
• config-file, containing the pathname of the dewarper configuration file
• Output
• Dewarped RGBA surfaces
• NvDewarperSurfaceMeta with information associated with each surface (projection_type, surface_index, and source_id), and the number of valid dewarped surfaces in the buffer (num_filled_surfaces)
Features
The following table summarizes the features of the plugin.
Features of the Gst-nvdewarper plugin | ||
|---|---|---|
Feature | Description | Release |
Configure number of dewarped surfaces | Supports a maximum of four dewarper surfaces. | DS 3.0 |
Configure per-surface projection type | Currently supports FishPushBroom and FishVertRadCyd projections. | DS 3.0 |
Configure per-surface index | Surface index to be set in case of multiple surfaces having same projection type. | DS 3.0 |
Configure per-surface width and height | DS 3.0 | |
Configure per-surface dewarping parameters | Per surface configurable yaw, roll, pitch, top angle, bottom angle, and focal length dewarping parameters. | DS 3.0 |
Configurable dewarper output resolution | Creates a batch of up to four surfaces of a specified output resolution; internally scales all dewarper surfaces to output resolution. | DS 3.0 |
Configurable NVDS CUDA memory type | — | DS 3.0 |
Multi-GPU support | — | DS 3.0 |
Aisle view CSV calibration file support | If set, properties in the [surface<n>] group are ignored. | DS 3.0 |
Spot view CSV calibration file support | If set, properties in the [surface<n>] group are ignored. | DS 3.0 |
Configure source id | Sets the source ID information in the NvDewarperSurfaceMeta. | DS 4.0 |
Configurable number of output buffers | Number of allocated output dewarper buffers. Each buffer contains four dewarped output surfaces. | DS 4.0 |
Configuration File Parameters
The configuration file specifies per-surface configuration parameters in [surface<n>] groups, where <n> is an integer from 0 to 3, representing dewarped surfaces 0 to 3.
Gst-nvdewarper plugin, configuration file, [surface<n>] parameters | |||
|---|---|---|---|
Property | Meaning | Type and Range | Example Notes |
output-width | Scale dewarped surfaces to specified output width | Integer, >0 | output-width=960 |
output-height | Scale dewarped surfaces to specified output height | Integer, >0 | output-height=752 |
dewarp-dump-frames | Number of dewarped frames to dump. | Integer, >0 | dewarp-dump-frames=10 |
projection-type | Selects projection type. Supported types are: 1: PushBroom 2: VertRadCyl | Integer, 1 or 2 | projection-type=1 |
surface-index | An index that distinguishes surfaces of the same projection type. | Integer, ≥0 | surface-index=0 |
width | Dewarped surface width. | Integer, >0 | width=3886 |
height | Dewarped surface height. | Integer, >0 | height=666 |
top-angle | Top field of view angle, in degrees. | Float, −180.0 to 180.0 | top-angle=0 |
bottom-angle | Bottom field of view angle, in degrees. | Float, −180.0 to 180.0 | bottom-angle=0 |
pitch | Viewing parameter pitch in degrees. | Float, 0.0 to 360.0 | pitch=90 |
yaw | Viewing parameter yaw in degrees. | Float, 0.0 to 360.0 | yaw=0 |
roll | Viewing parameter roll in degrees. | Float, 0.0 to 360.0 | roll=0 |
focal-length | Focal length of camera lens, in pixels per radian. | Float, >0.0 | focal-length=437 |
aisle-calibration-file | Pathname of the configuration file for aisle view. Set for the 360‑D application only. If set, properties in the [surface<n>] group are ignored. The configuration file is a CSV file with columns like sensorId and cameraId, and dewarping parameters like top-angle, bottom-angle, yaw, roll, pitch, focal-length, width, and height. | String | aisle-calibration-file=csv_files/nvaisle_2M.csv |
spot-calibration-file | Pathname of the configuration file for spot view. Set for the 360‑D application only. If set, properties in the [surface<n>] group are ignored. The configuration file is a CSV file with columns like sensorId and cameraId, and dewarping parameters like top-angle, bottom-angle, yaw, roll, pitch, focal-length, width, and height. | String | spot-calibration-file=csv_files/nvspot_2M.csv For an example of a spot view configuration file, see the file in the example above. |
This plugin can be tested with the one of the following pipelines.
• For dGPU:
gst-launch-1.0 filesrc location=streams/sample_cam6.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! nvvideoconvert ! nvdewarper config-file=config_dewarper.txt source-id=6 nvbuf-memory-type=3 ! m.sink_0 nvstreammux name=m width=960 height=752 batch-size=4 num-surfaces-per-frame=4 ! nvmultistreamtiler ! nveglglessink
• For Jetson:
gst-launch-1.0 filesrc location= streams/sample_cam6.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! nvvideoconvert ! nvdewarper config-file=config_dewarper.txt source-id=6 ! m.sink_0 nvstreammux name=m width=960 height=752 batch-size=4 num-surfaces-per-frame=4 ! nvmultistreamtiler ! nvegltransform ! nveglglessink
The Gst-nvdewarper plugin always outputs a GStreamer buffer which contains the maximum number of dewarped surfaces (currently four surfaces are supported). These dewarped surfaces are scaled to the output resolution (output-width × output-height) set in the configuration file located at configs/deepstream-app/config_dewarper.txt.
Also, the batch size to be set on Gst-nvstreammux must be a multiple of the maximum number of dewarped surfaces (currently four).
Gst Properties
The following table describes the Gst-nvdewarper plugin’s Gst properties.
Gst-nvdewarper plugin, Gst properties | |||
|---|---|---|---|
Property | Meaning | Type and Range | Example and Notes |
config-file | Absolute pathname of configuration file for the Gst-nvdewarper element | String | config-file= configs/ deepstream-app/ config_dewarper.txt |
gpu-id | Device ID of the GPU to be used (dGPU only) | Integer, 0 to 4,294,967,295 | gpu-id=0 |
source-id | Source ID, e.g. camera ID | Integer, 0 to 4,294,967,295 | source-id=6 |
num-output-buffers | Number of output buffers to be allocated | Integer, 0 to 4,294,967,295 | num-output-buffers=4 |
nvbuf-memory-type | Type of memory to be allocated. For dGPU: 0 (nvbuf-mem-default): Default memory, cuda-device 1 (nvbuf-mem-cuda-pinned): Pinned/Host CUDA memory 2 (nvbuf-mem-cuda-device) Device CUDA memory 3 (nvbuf-mem-cuda-unified): Unified CUDA memory For Jetson: 0 (nvbuf-mem-default): Default memory, surface array 4 (nvbuf-mem-surface-array): Surface array memory | Integer, 0 to 4 | nvbuf-memory-type=3 |