Image OpticalFlow/StereoDisparity (nvmedia_imageofst)

The NvMedia nvmedia_imageofst sample application demonstrates how to use the NvMedia Optical Flow or Stereo Disparity (OFST) APIs to perform motion estimation/stereo matching. The sample uses the NVIDIA® NVENC hardware engine.
nvmedia_imageofst can read YUV files and perform motion estimation/stereo matching on input YUV files. The sample outputs one motion vector for each 4×4 input block on the output surface.
The output motion vector format is fixed signed 10.5 [Range -1024 to 1023] with quarter-pixel accuracy [the last 3 bits of fractional part are always zero].
For Optical Flow, we need to divide MVx and MVy values by 32 to get the flow vector in terms of pixel units.
For Stereo Disparity, the output corresponds to the left view, and the absolute value of the output provides the actual disparity. We need to divide the output values by 32 to get a disparity in terms of pixel units.

Architecture

This figure shows the architecture of the nvmedia_imageofst application.
 
A screenshot of a video game Description generated with high confidence
To run the sample
2. Launch the application:
$ nvmedia_imageofst -h -v 3

Command Line Options

The sample application command syntax is as follows:
$ nvmedia_imageofst [options]
The command line options are as follows.
String parameters, such as pathnames, have a maximum length of 1024 characters. Restrictions on other types of parameters are noted individually.
Required Command Line Options
Option
Parameter
Description
-f
A pathname
Specifies the full pathname of the input file.
-res
<w>x<h>
Specifies the input surface dimensions, where <w> represents width and <h> represents height. Width and height are separated by a letter ‘x’ with no spaces, for example:
-res 640x480
<w> has a range of [160,3840].
<h> has a range of [64,2160].
-o
A file name
Specifies the full pathname of the output MV file.
Output MV file.
 
The output surface contains 4×4 downsampled MV’s. The size of each MV is:
Optical Flow: 4 bytes (MVx,MVy)
Stereo Disparity: 2 bytes (MVx)
Optional Command Line Options
Option
Parameter
Description
-h
n/a
Displays guidance on using this application.
-v
0: Errors (default)
1: Warnings
2: Info
3: Debug
Logging level.
-crcgen
A pathname of a file to be created with the filetype .crc
Generates CRC values in the specified file.
-crcchk
A pathname of an existing file with the filetype .crc
Checks CRC values against values from the specified file.
-hwid
0 (default)
1
Specifies the nvenc instance ID to use for motion estimation/stereo matching.
-if
0: YV12 (default)
1: IYUV
Input YUV file format.
-frames
An integer
Specifies the number of input frames to be processed
-etype
0: opticalFlow estimation high performance (default)
1: stereoDisparity estimation high performance
2: opticalFlow estimation high quality
3: stereoDisparity estimation high quality
4: opticalFlow estimation ultra-high performance
5: stereoDisparity estimation ultra-high performance
Specifies motion estimation/stereo matching mode.
-ehints
A pathname of an existing file
Specifies the external hints file.
-esegmap
n/a
Enables the segmentation map.
-bit_depth
8: 8 bits (default)
10: 10 bits
12: 12 bits
14: 14 bits
16: 16 bits
Number of bits per components on input surfaces.
-cf_idc
1: 420 (default)
2: 422
3: 444
4: 400
YUV subsampling type of input file.
-forward_ref
n/a
Specifies that the reference frame is forward reference. The default is backward reference. For Stereo Disparity, this option should be enabled to get the proper output.

Examples

These examples demonstrate how to call nvmedia_imageofst to perform various operations.
High performance optical flow estimation using backward reference
$ nvmedia_imageofst -f inputfile -res wxh -o outputfile -etype 0 -frames 20 -v 3
The application performs motion estimation according to the parameters in the command line.
High quality optical flow estimation using forward reference
$ nvmedia_imageofst -f inputfile -res wxh -o outputfile -etype 2 -forward_ref -frames 20 -v 3
The application performs motion estimation according to the parameters in the command line.
High quality Stereo Disparity estimation for YUV subsampling type 444 10-bit input
$ nvmedia_imageofst -f inputfile -res wxh -o outputfile -etype 3 -forward_ref -bit_depth 10 -cf_idc 3 -frames 20 -v 3
The application performs stereo matching according to the parameters in the command line.
Ultra-high performance Stereo Disparity estimation for YUV subsampling type 422 16-bit input
$ nvmedia_imageofst -f inputfile -res wxh -o outputfile -etype 5 -forward_ref -bit_depth 16 -cf_idc 2 -frames 20 -v 3
The application performs stereo matching according to the parameters in the command line.
Ultra-high performance Optical Flow estimation with CRC generation
$ nvmedia_imageofst -f inputfile -res wxh -o outputfile -etype 4 -crcgen crcfile -frames 20 -v 3
The application performs motion estimation according to the parameters in the command line.