Image OpticalFlow/StereoDisparity (OFST) Estimator

Detailed Description

The NvMediaIOFST object takes an uncompressed image frame pair and turns them into opticalflow/stereodisparity estimation data.

Data Structures
struct	NvMediaIOFST
	Holds an OFST object created by NvMediaIOFSTCreate. More...
struct	NvMediaOFSTInitializeParams
	Holds OFST estimation initialization parameters. More...
struct	NvMediaOFSTExternalHintParams
	Holds OFST estimation parameters. More...

Macros
#define	NVMEDIA_IOFST_VERSION_MAJOR   1
	Major version number. More...
#define	NVMEDIA_IOFST_VERSION_MINOR   14
	Minor version number. More...

Enumerations
enum	NvMediaIOFSTType {   NVMEDIA_IMAGE_OPTICALFLOW_ESTIMATION_HP_MODE,   NVMEDIA_IMAGE_STEREODISPARITY_ESTIMATION_HP_MODE,   NVMEDIA_IMAGE_OPTICALFLOW_ESTIMATION_HQ_MODE,   NVMEDIA_IMAGE_STEREODISPARITY_ESTIMATION_HQ_MODE,   NVMEDIA_IMAGE_OPTICALFLOW_ESTIMATION_UHP_MODE,   NVMEDIA_IMAGE_STEREODISPARITY_ESTIMATION_UHP_MODE }
	Defines the image estimation type. More...
enum	NvMediaOFSTConfigFeatures { NVMEDIA_OFST_CONFIG_ENABLE_PROFILING = (1 << 0) }
	Defines OFST estimation configuration features. More...

Functions
NvMediaStatus	NvMediaIOFSTGetVersion (NvMediaVersion *version)
	Returns the version information for the NvMedia IOFST library. More...
NvMediaIOFST *	NvMediaIOFSTCreate (const NvMediaDevice *device, NvMediaIOFSTType estimationType, NvMediaSurfaceType inputFormat, NvMediaSurfaceType outputFormat, const NvMediaOFSTInitializeParams *initParams, uint8_t maxInputBuffering, NvMediaEncoderInstanceId instanceId)
	Creates an OFST object that can create motion vectors based on the difference between two frames. More...
void	NvMediaIOFSTDestroy (const NvMediaIOFST *ofst)
	Destroys an NvMediaIOFST object. More...
NvMediaStatus	NvMediaIOFSTProcessFrame (const NvMediaIOFST *ofst, const NvMediaImage *frame, const NvMediaImage *refFrame, const NvMediaImage *mvs, const NvMediaOFSTExternalHintParams *extHintParams, NvMediaEncoderInstanceId instanceId)
	Performs OFST estimation on a specified frame pair. More...
NvMediaStatus	NvMediaIOFSTImageRegister (const NvMediaIOFST *ofst, const NvMediaImage *image, NvMediaAccessMode accessMode)
	Registers NvMediaImage for use with a NvMediaIOFST handle. More...
NvMediaStatus	NvMediaIOFSTImageUnRegister (const NvMediaIOFST *ofst, const NvMediaImage *image)
	Un-registers NvMediaImage which is registered with NvMediaIOFST. More...

Macro Definition Documentation

NVMEDIA_IOFST_VERSION_MAJOR

#define NVMEDIA_IOFST_VERSION_MAJOR   1

Major version number.

Definition at line 37 of file nvmedia_iofst.h.

NVMEDIA_IOFST_VERSION_MINOR

#define NVMEDIA_IOFST_VERSION_MINOR   14

Minor version number.

Definition at line 39 of file nvmedia_iofst.h.

Enumeration Type Documentation

NvMediaIOFSTType

enum NvMediaIOFSTType

Defines the image estimation type.

Enumerator
NVMEDIA_IMAGE_OPTICALFLOW_ESTIMATION_HP_MODE	High Performance OpticalFlow.
NVMEDIA_IMAGE_STEREODISPARITY_ESTIMATION_HP_MODE	High Performance StereoDisparity.
NVMEDIA_IMAGE_OPTICALFLOW_ESTIMATION_HQ_MODE	High Quality OpticalFlow.
NVMEDIA_IMAGE_STEREODISPARITY_ESTIMATION_HQ_MODE	High Quality StereoDisparity.
NVMEDIA_IMAGE_OPTICALFLOW_ESTIMATION_UHP_MODE	Ultra High Performance OpticalFlow.
NVMEDIA_IMAGE_STEREODISPARITY_ESTIMATION_UHP_MODE	Ultra High Performance StereoDisparity.

Definition at line 44 of file nvmedia_iofst.h.

NvMediaOFSTConfigFeatures

enum NvMediaOFSTConfigFeatures

Defines OFST estimation configuration features.

Enumerator
NVMEDIA_OFST_CONFIG_ENABLE_PROFILING	Enables OFST profiling.

Definition at line 62 of file nvmedia_iofst.h.

Function Documentation

NvMediaIOFSTCreate()

NvMediaIOFST* NvMediaIOFSTCreate	(	const NvMediaDevice *	device,
		NvMediaIOFSTType	estimationType,
		NvMediaSurfaceType	inputFormat,
		NvMediaSurfaceType	outputFormat,
		const NvMediaOFSTInitializeParams *	initParams,
		uint8_t	maxInputBuffering,
		NvMediaEncoderInstanceId	instanceId
	)

Creates an OFST object that can create motion vectors based on the difference between two frames.

Parameters

[in]	device	A pointer to the NvMediaDevice this OFST is to use. Range: Non-null - valid address
[in]	estimationType	OF/ST estimation type. Range: NVMEDIA_IMAGE_OPTICALFLOW_ESTIMATION_HP_MODE NVMEDIA_IMAGE_STEREODISPARITY_ESTIMATION_HP_MODE NVMEDIA_IMAGE_OPTICALFLOW_ESTIMATION_HQ_MODE NVMEDIA_IMAGE_STEREODISPARITY_ESTIMATION_HQ_MODE NVMEDIA_IMAGE_OPTICALFLOW_ESTIMATION_UHP_MODE NVMEDIA_IMAGE_STEREODISPARITY_ESTIMATION_UHP_MODE
[in]	inputFormat	Input format. Range: Input format returned by a call to NvMediaSurfaceFormatGetType() with the following for safety build: NVM_SURF_FMT_SET_ATTR_YUV(attr, YUV, 420, SEMI_PLANAR, UINT, [8/16], BL) NvMediaSurfaceFormatGetType() with the following for non-safety build: NVM_SURF_FMT_SET_ATTR_YUV(attr, LUMA, NONE, PACKED, UINT, [8/10/16], BL) NVM_SURF_FMT_SET_ATTR_YUV(attr, YUV, 420, SEMI_PLANAR, UINT, [8/10/12/16], BL) NVM_SURF_FMT_SET_ATTR_YUV(attr, YUV, 444, SEMI_PLANAR, UINT, [8/10/12/16], BL)
[in]	outputFormat	Output format. Range: Obtained by a call to NvMediaSurfaceFormatGetType() with: NVM_SURF_FMT_SET_ATTR_RGBA(attr, RG, INT, 16, BL) for Optical Flow NVM_SURF_FMT_SET_ATTR_RGBA(attr, ALPHA, INT, 16, BL) for Stereo Disparity
[in]	initParams	A pointer to a structure that specifies initialization parameters. Range: Non-null - valid address. Ranges specific to each member in the structure can be found in NvMediaOFSTInitializeParams
[in]	maxInputBuffering	Number of NvMediaIOFSTProcessFrame() operations that can be queued by the NvMediaIOFST. If more than maxInputBuffering operations are queued, NvMediaIOFSTProcessFrame() blocks until the excess operations are dequeued. Range: 1-16 (steps of 1)
[in]	instanceId	ID of the encoder engine instance. Range: NVMEDIA_ENCODER_INSTANCE_0 NVMEDIA_ENCODER_INSTANCE_1 NVMEDIA_ENCODER_INSTANCE_AUTO [No support in Safety Build]

Returns

The new NvM OFST Estimator handle if successful, or NULL otherwise.

NvMediaIOFSTDestroy()

void NvMediaIOFSTDestroy

(

const NvMediaIOFST *

ofst

)

Destroys an NvMediaIOFST object.

Parameters

[in]

ofst

The OFST object to destroy. Range: Non-null – valid pointer address

NvMediaIOFSTGetVersion()

NvMediaStatus NvMediaIOFSTGetVersion

(

NvMediaVersion *

version

)

Returns the version information for the NvMedia IOFST library.

Parameters

[out]

version

: A pointer to a NvMediaVersion structure filled by the IOFST library.

Returns

NvMediaStatus, the completion status of the operation: NVMEDIA_STATUS_OK if successful, or NVMEDIA_STATUS_BAD_PARAMETER if the pointer is invalid.

NvMediaIOFSTImageRegister()

NvMediaStatus NvMediaIOFSTImageRegister	(	const NvMediaIOFST *	ofst,
		const NvMediaImage *	image,
		NvMediaAccessMode	accessMode
	)

Registers NvMediaImage for use with a NvMediaIOFST handle.

NvMediaIOFST handle maintains a record of all the images registered using this API.

This is an optional API, skipping it will result in non-deterministic NvMediaIOFSTProcessFrame execution time.

For deterministic execution time of NvMediaIOFSTProcessFrame API :

• NvMediaIOFSTImageRegister must be called for every input and output NvMediaImage that will be used with NvMediaIOFST
• All NvMediaIOFSTImageRegister calls must be made before first NvMediaIOFSTProcessFrame API call.

Maximum of 32 NvMediaImage handles can be registered per access mode.

Parameters

[in]	ofst	: NvMedia IOFST device handle.
[in]	image	: A pointer to NvMedia image
[in]	accessMode	: NvMediaAccessMode required for the image

Returns

NvMediaStatus, the completion status of operation:

• NVMEDIA_STATUS_OK if successful.
• NVMEDIA_STATUS_BAD_PARAMETER if ofst, image or accessMode is invalid
• NVMEDIA_STATUS_ERROR in following cases
  • user registers more than 32 images.
  • user registers same image with more than one accessModes.
  • user registers same image multiple times
• NVMEDIA_STATUS_NOT_SUPPORTED if API is not functional

NvMediaIOFSTImageUnRegister()

NvMediaStatus NvMediaIOFSTImageUnRegister	(	const NvMediaIOFST *	ofst,
		const NvMediaImage *	image
	)

Un-registers NvMediaImage which is registered with NvMediaIOFST.

For all NvMediaImage handles registered with NvMediaIOFST using NvMediaIOFSTImageRegister API, NvMediaIOFSTImageUnRegister must be called before calling NvMediaIOFSTDestroy API. For unregistration to succeed, it should be ensured that none of the submitted tasks on the image are pending prior to calling NvMediaIOFSTImageUnRegister API. In order to ensure this, NvMediaImageGetStatus API needs to be called on all output NvMediaImage, to wait for their task completion. Post this NvMediaIOFSTImageUnRegister can be successfully called on valid NvMediaImage

For deterministic execution of NvMediaIOFSTProcessFrame API, NvMediaIOFSTImageUnRegister must be called only after last NvMediaIOFSTProcessFrame call.

Parameters

[in]	ofst	: NvMedia IOFST device handle.
[in]	image	: A pointer to NvMedia image

Returns

NvMediaStatus, the completion status of operation:

• NVMEDIA_STATUS_OK if successful.
• NVMEDIA_STATUS_BAD_PARAMETER if ofst is invalid or unregister is called for image which is not registered using NvMediaIOFSTImageRegister API
• NVMEDIA_STATUS_NOT_SUPPORTED if API is not functional
• NVMEDIA_STATUS_PENDING if the enqueued operations on the NvMediaImage are still pending. In this case, the application can choose to wait for operations to complete on the output surface using NvMediaImageGetStatus or re-try the NvMediaIOFSTImageUnRegister API call, untill the status returned is not NVMEDIA_STATUS_PENDING

NvMediaIOFSTProcessFrame()

NvMediaStatus NvMediaIOFSTProcessFrame	(	const NvMediaIOFST *	ofst,
		const NvMediaImage *	frame,
		const NvMediaImage *	refFrame,
		const NvMediaImage *	mvs,
		const NvMediaOFSTExternalHintParams *	extHintParams,
		NvMediaEncoderInstanceId	instanceId
	)

Performs OFST estimation on a specified frame pair.

Estimation is based on the difference between refFrame and frame. The OFST produces motion vectors.

Parameters

[in]	ofst	A pointer to the OFSTestimator to use. Range: Non-null – valid pointer address
[in]	frame	A pointer to current frame YUV data. It must be the same sourceType as ofst->inputformat. Range: Non-null – valid pointer address. For Stereo Disparity this specifies the handle to the buffer containing the rectified left view.
[in]	refFrame	A pointer to reference frame YUV data. It must be the same sourceType as ofst->inputformat. Range: Non-null – valid pointer address. For Stereo Disparity this specifies the handle to the buffer containing the rectified right view.
[in]	mvs	A pointer to a frame which contains motion vector output. It must be of the same sourceType as ofst->outputformat. Range: Non-null – valid pointer address. Motion vector output is in signed 11.5 format and quarter pixel accuracy. Which means last 3 bits of fractional part are zero all the time. For Stereo Disparity, output corresponds to left view and absolute value of output will provide actual disparity.
[in]	extHintParams	A pointer to a structure containing external hint parameters. It is used for ME hinting. Set it to NULL if ME hinting is not needed.External hint parameter is not supported in safety build. Range: NULL to disable ME hints, Non-null valid pointer address to enable ME hints
[in]	instanceId	The ID of the encoder engine instance. In the case of non-safety builds, if NVMEDIA_ENCODER_INSTANCE_AUTO is used in the call to NvMediaVideoEncoderCreate(), the following instances are supported: - NVMEDIA_ENCODER_INSTANCE_0 - NVMEDIA_ENCODER_INSTANCE_1 Otherwise an error is thrown.

Returns

The completion status of the operation: NVMEDIA_STATUS_OK if the call is successful, or NVMEDIA_STATUS_BAD_PARAMETER if any of the input parameters are invalid.