Gst-nvof

NVIDIA DeepStream Plugin Manual

Application Note

4.0.1 Release

Gst-nvof

NVIDIA GPUs, starting with the dGPU Turing generation and Jetson Xavier generation, contain a hardware accelerator for computing optical flow. Optical flow vectors are useful in various use cases such as object detection and tracking, video frame rate up-conversion, depth estimation, stitching, and so on.

The gst-nvof plugin collects a pair of NV12 images and passes it to the low-level optical flow library. The low-level library returns a map of flow vectors between the two frames as its output.

The map of flow vectors is encapsulated in the NvDsOpticalFlowMeta structure and is added as a user meta with meta_type set to NVDS_OPTICAL_FLOW_META. The user meta is added to the frame_user_meta_list member of NvDsFrameMeta.

For guidance on how to access user metadata, see User/Custom Metadata Addition Inside NvDsMatchMeta and Tensor Metadata.

Inputs and Outputs

• Inputs

• GStreamer buffer containing NV12 frame(s)

• Control parameters

• gpu-id: selects the GPU ID (valid only for dGPU platforms)

• dump-of-meta: enables dumping of optical flow map vector into a .bin file

• preset-level: sets the preset level

• pool-size: sets the pool size

• grid-size: sets the grid size

• Outputs

• GStreamer buffer containing NV12 frame(s)

• NvDsOpticalFlowMeta metadata

Features

The following table summarizes the features of the plugin.

Features of the Gst-nvof plugin
Feature	Description	Release
Configure GPU selection	Sets the gpu ID to be used for optical flow operation (valid only for dGPU platforms)	DS 4.0
Configure dumping of optical flow metadata	Enables dumping of optical flow output (motion vector data)	DS 4.0
Configure preset level	Sets the desired preset level	DS 4.0
Configure grid size	Sets the flow vector block size	DS 4.0

Gst Properties

The following table describes the Gst properties of the Gst-nvof plugin.

Gst-nvof plugin, Gst properties
Property	Meaning	Type and Range	Example Notes
gpu-id	Device ID of the GPU to be used for decoding (dGPU only).	Integer, 0 to 4,294,967,295	gpu-id=0
dump-of-meta	Dumps optical flow output into a .bin file.	Boolean	dump-of-meta=1
preset-level	Selects a preset level, default preset level is 0 i.e. NV_OF_PERF_LEVEL_FAST Possible values are: 0 (NV_OF_PERF_LEVEL_FAST): high performance, low quality. 1 (NV_OF_PERF_LEVEL_MEDIUM): intermediate performance and quality. 2 (NV_OF_PERF_LEVEL_SLOW): low performance, best quality (valid only for dGPU platforms).	Enum, 0 to 2	preset-level=0
grid-size	Selects the grid size. The hardware generates flow vectors blockwise, one vector for each block of 4×4 pixels. Currently only the 4x4 grid size is supported.	Enum, 0	grid-size=0
pool-size	Sets the number of internal motion vector output buffers to be allocated.	Integer, 1 to 4,294,967,295	pool-size=7