The NvMediaIOFST object takes an uncompressed image frame pair and turns them into opticalflow/stereodisparity estimation data.
Data Structures | |
struct | NvMediaIOFST |
Holds an OFST object created and returned by NvMediaIOFSTCreate(). More... | |
struct | NvMediaOFSTInitializeParams |
Holds OFST estimation initialization parameters. More... | |
struct | NvMediaOFSTExternalHintParams |
Holds OFST estimation parameters. More... | |
Macros | |
#define | NVMEDIA_IOFST_VERSION_MAJOR 1 |
Major version number. More... | |
#define | NVMEDIA_IOFST_VERSION_MINOR 14 |
Minor version number. More... | |
Enumerations | |
enum | NvMediaIOFSTType { NVMEDIA_IMAGE_OPTICALFLOW_ESTIMATION_HP_MODE, NVMEDIA_IMAGE_STEREODISPARITY_ESTIMATION_HP_MODE, NVMEDIA_IMAGE_OPTICALFLOW_ESTIMATION_HQ_MODE, NVMEDIA_IMAGE_STEREODISPARITY_ESTIMATION_HQ_MODE, NVMEDIA_IMAGE_OPTICALFLOW_ESTIMATION_UHP_MODE, NVMEDIA_IMAGE_STEREODISPARITY_ESTIMATION_UHP_MODE } |
Defines the image estimation type. More... | |
enum | NvMediaOFSTConfigFeatures { NVMEDIA_OFST_CONFIG_ENABLE_PROFILING = (1 << 0) } |
Defines OFST estimation configuration features. More... | |
Functions | |
NvMediaStatus | NvMediaIOFSTGetVersion (NvMediaVersion *version) |
Returns the version information for the NvMedia IOFST library. More... | |
NvMediaIOFST * | NvMediaIOFSTCreate (const NvMediaDevice *device, NvMediaIOFSTType estimationType, NvMediaSurfaceType inputFormat, NvMediaSurfaceType outputFormat, const NvMediaOFSTInitializeParams *initParams, uint8_t maxInputBuffering, NvMediaEncoderInstanceId instanceId) |
Creates an NvMediaIOFST object that can compute motion vectors or a disparity map based on differences between two images. More... | |
void | NvMediaIOFSTDestroy (const NvMediaIOFST *ofst) |
Destroys the created NvMediaIOFST object and frees associated resources. More... | |
NvMediaStatus | NvMediaIOFSTProcessFrame (const NvMediaIOFST *ofst, const NvMediaImage *frame, const NvMediaImage *refFrame, const NvMediaImage *mvs, const NvMediaOFSTExternalHintParams *extHintParams, NvMediaEncoderInstanceId instanceId) |
Performs OF/ST estimation on a specified frame pair. More... | |
NvMediaStatus | NvMediaIOFSTImageRegister (const NvMediaIOFST *ofst, const NvMediaImage *image, NvMediaAccessMode accessMode) |
Registers an NvMediaImage for use with an NvMediaIOFST handle. More... | |
NvMediaStatus | NvMediaIOFSTImageUnRegister (const NvMediaIOFST *ofst, const NvMediaImage *image) |
Un-registers NvMediaImage which was previously registered with NvMediaIOFST using NvMediaIOFSTImageRegister(). More... | |
#define NVMEDIA_IOFST_VERSION_MAJOR 1 |
Major version number.
Definition at line 38 of file nvmedia_iofst.h.
#define NVMEDIA_IOFST_VERSION_MINOR 14 |
Minor version number.
Definition at line 40 of file nvmedia_iofst.h.
enum NvMediaIOFSTType |
Defines the image estimation type.
Definition at line 45 of file nvmedia_iofst.h.
Defines OFST estimation configuration features.
Enumerator | |
---|---|
NVMEDIA_OFST_CONFIG_ENABLE_PROFILING | Enables OFST profiling.
|
Definition at line 81 of file nvmedia_iofst.h.
NvMediaIOFST* NvMediaIOFSTCreate | ( | const NvMediaDevice * | device, |
NvMediaIOFSTType | estimationType, | ||
NvMediaSurfaceType | inputFormat, | ||
NvMediaSurfaceType | outputFormat, | ||
const NvMediaOFSTInitializeParams * | initParams, | ||
uint8_t | maxInputBuffering, | ||
NvMediaEncoderInstanceId | instanceId | ||
) |
Creates an NvMediaIOFST object that can compute motion vectors or a disparity map based on differences between two images.
Considerations for Safety:
[in] | device | A pointer to the NvMediaDevice this OFST is to use. Input range: Non-NULL - valid address |
[in] | estimationType | OF/ST estimation type. Input range: One of the following: NVMEDIA_IMAGE_OPTICALFLOW_ESTIMATION_HP_MODE NVMEDIA_IMAGE_STEREODISPARITY_ESTIMATION_HP_MODE NVMEDIA_IMAGE_OPTICALFLOW_ESTIMATION_HQ_MODE NVMEDIA_IMAGE_STEREODISPARITY_ESTIMATION_HQ_MODE NVMEDIA_IMAGE_OPTICALFLOW_ESTIMATION_UHP_MODE NVMEDIA_IMAGE_STEREODISPARITY_ESTIMATION_UHP_MODE |
[in] | inputFormat | Format of the input NvMediaImage. Input range: Input format returned by a call to NvMediaSurfaceFormatGetType(), with attr being set as follows: QNX Safety build: NVM_SURF_FMT_SET_ATTR_YUV(attr, YUV, 420, SEMI_PLANAR, UINT, [8/16], BL) Non-safety build: NVM_SURF_FMT_SET_ATTR_YUV(attr, LUMA, NONE, PACKED, UINT, [8/10/16], BL) NVM_SURF_FMT_SET_ATTR_YUV(attr, YUV, 420, SEMI_PLANAR, UINT, [8/10/12/16], BL) NVM_SURF_FMT_SET_ATTR_YUV(attr, YUV, 444, SEMI_PLANAR, UINT, [8/10/12/16], BL) |
[in] | outputFormat | Format of the output NvMediaImage. Input range: Output format obtained by a call to NvMediaSurfaceFormatGetType(), with attr being set as follows: NVM_SURF_FMT_SET_ATTR_RGBA(attr, RG, INT, 16, BL) for Optical Flow NVM_SURF_FMT_SET_ATTR_RGBA(attr, ALPHA, INT, 16, BL) for Stereo Disparity |
[in] | initParams | A pointer to a structure that specifies initialization parameters. Input range: Non-NULL - valid address. Ranges specific to each member in the structure can be found in NvMediaOFSTInitializeParams. |
[in] | maxInputBuffering | Maximum number of NvMediaIOFSTProcessFrame() operations that can be queued by NvMediaIOFST. If more than maxInputBuffering operations are queued, NvMediaIOFSTProcessFrame() returns an error to indicate insufficient buffering. Input range: The values between 1 to 16, in increments of 1 |
[in] | instanceId | ID of the encoder engine instance. Input range: One of the following: NVMEDIA_ENCODER_INSTANCE_0 [Supported only on Non-safety builds] NVMEDIA_ENCODER_INSTANCE_1 NVMEDIA_ENCODER_INSTANCE_AUTO [Supported only on Non-safety builds] |
void NvMediaIOFSTDestroy | ( | const NvMediaIOFST * | ofst | ) |
Destroys the created NvMediaIOFST object and frees associated resources.
Considerations for Safety:
[in] | ofst | The NvMediaIOFST object to destroy, returned by NvMediaIOFSTCreate(). Input range: Non-NULL - valid pointer address |
NvMediaStatus NvMediaIOFSTGetVersion | ( | NvMediaVersion * | version | ) |
Returns the version information for the NvMedia IOFST library.
Considerations for Safety:
[out] | version | A pointer to a NvMediaVersion structure filled by the IOFST library. |
NvMediaStatus NvMediaIOFSTImageRegister | ( | const NvMediaIOFST * | ofst, |
const NvMediaImage * | image, | ||
NvMediaAccessMode | accessMode | ||
) |
Registers an NvMediaImage for use with an NvMediaIOFST handle.
The NvMediaIOFST handle maintains a record of all the images registered using this API.
This is a mandatory API in safety build as skipping it will result in non-deterministic NvMediaIOFSTProcessFrame() execution time.
For deterministic execution time of NvMediaIOFSTProcessFrame() API :
Maximum of 32 NvMediaImage handles can be registered per access mode.
Considerations for Safety:
[in] | ofst | NvMediaIOFST handle. Input range: Non-NULL - valid pointer address |
[in] | image | A pointer to NvMediaImage object. Input range: Non-NULL - valid pointer address |
[in] | accessMode | NvMediaAccessMode required for the image. Input range: One of the following based on the type of NvMediaImage being registered:
|
NvMediaStatus NvMediaIOFSTImageUnRegister | ( | const NvMediaIOFST * | ofst, |
const NvMediaImage * | image | ||
) |
Un-registers NvMediaImage which was previously registered with NvMediaIOFST using NvMediaIOFSTImageRegister().
For all NvMediaImage handles registered with NvMediaIOFST using NvMediaIOFSTImageRegister() API, NvMediaIOFSTImageUnRegister() must be called before calling NvMediaIOFSTDestroy() API. For unregistration to succeed, it should be ensured that none of the submitted tasks on the image are pending prior to calling NvMediaIOFSTImageUnRegister() API. In order to ensure this, NvMediaImageGetStatus() API needs to be called on all output NvMediaImage, to wait for their task completion. Post this NvMediaIOFSTImageUnRegister() can be successfully called on valid NvMediaImage.
For deterministic execution of NvMediaIOFSTProcessFrame() API, NvMediaIOFSTImageUnRegister() must be called only after last NvMediaIOFSTProcessFrame() call.
Considerations for Safety:
[in] | ofst | NvMediaIOFST handle. Input range: Non-NULL - valid pointer address |
[in] | image | A pointer to NvMediaImage to unregister. Input range: Non-NULL - valid pointer address |
NvMediaStatus NvMediaIOFSTProcessFrame | ( | const NvMediaIOFST * | ofst, |
const NvMediaImage * | frame, | ||
const NvMediaImage * | refFrame, | ||
const NvMediaImage * | mvs, | ||
const NvMediaOFSTExternalHintParams * | extHintParams, | ||
NvMediaEncoderInstanceId | instanceId | ||
) |
Performs OF/ST estimation on a specified frame pair.
Estimation is based on the difference between refFrame and frame. The output of Optical Flow processing is motion vectors, and that of Stereo Disparity processing is a disparity surface.
Considerations for Safety:
[in] | ofst | A pointer to the NvMediaIOFST estimator to use. Input range: Non-NULL - valid pointer address |
[in] | frame | Optical Flow Processing: A pointer to an input NvMediaImage at time T+1 (current frame). Stereo Disparity Processing: A pointer to an input NvMediaImage containing the rectified left view. It must be the same sourceType as ofst->inputformat. frame is allocated through a call to NvMediaImageCreateFromNvSciBuf(). Input range: Non-NULL - valid pointer address. |
[in] | refFrame | Optical Flow Processing: A pointer to an input NvMediaImage at time T, which acts as a reference to current frame. Stereo Disparity Processing: A pointer to an input NvMediaImage containing the rectified right view. It must be the same sourceType as ofst->inputFormat. refFrame is allocated through a call to NvMediaImageCreateFromNvSciBuf(). Input range: Non-NULL - valid pointer address. |
[in] | mvs | A pointer to an NvMediaImage where motion vector output (for Optical Flow) and disparity surface (for Stereo Disparity) will be populated as a result of OF/ST processing. As the processing in not synchronous mvs will not be populated as soon as this API returns. Clients can optionally call NvMediaImageGetStatus() API to know the status of processing once this API returns. mvs must be of the same sourceType as ofst->outputFormat. mvs is allocated through a call to NvMediaImageCreateFromNvSciBuf(). Motion vector output is in S10.5 format, with a range of [-1024, 1023] and quarter pixel accuracy. This means that the MSB represents the sign of the flow vector, next 10bits represent the integer value and the last 5 bits represent the fractional value. As we support quarter pixel precision, out of the 5 bits that represent the fractional component, only the most significant 2 bits are in use. The last 3 bits of the fractional part are zero all the time. For Stereo Disparity, output corresponds to left view and absolute value of output will provide actual disparity. Input range: Non-NULL - valid pointer address. |
[in] | extHintParams | A pointer to a structure containing external hint parameters corresponding to the input frame pair. This feature is not supported on QNX Safety build, the field should be set to NULL. Input range: NULL to disable ME hints Non-NULL - valid pointer address to enable ME hints |
[in] | instanceId | The ID of the encoder engine instance. Input range: In the case of QNX safety build, this needs to be set to the same value of instanceId passed to NvMediaIOFSTCreate(). In the case of non-safety builds, if instanceId passed to NvMediaIOFSTCreate() was not NVMEDIA_ENCODER_INSTANCE_AUTO, then the value of to instanceId be passed here, should be the same as that of the value passed to NvMediaIOFSTCreate(). If NVMEDIA_ENCODER_INSTANCE_AUTO was passed to NvMediaIOFSTCreate(), the following instances are supported: NVMEDIA_ENCODER_INSTANCE_0 NVMEDIA_ENCODER_INSTANCE_1 If any other value is passed, an error is returned. |