VPI - Vision Programming Interface

3.0 Release

Stereo Disparity Estimator

Estimates disparity from a stereo pair. More...

Data Structures

struct  VPIStereoDisparityEstimatorCreationParams
 Structure that defines the parameters for vpiCreateStereoDisparityEstimator. More...
 
struct  VPIStereoDisparityEstimatorParams
 Structure that defines the parameters for vpiSubmitStereoDisparityEstimator. More...
 

Enumerations

enum  VPIStereoDisparityConfidenceType
 Defines the way the confidence values are computed. More...
 

Functions

VPIStatus vpiInitStereoDisparityEstimatorCreationParams (VPIStereoDisparityEstimatorCreationParams *params)
 Initializes VPIStereoDisparityEstimatorCreationParams with default values. More...
 
VPIStatus vpiCreateStereoDisparityEstimator (uint64_t backends, int32_t imageWidth, int32_t imageHeight, VPIImageFormat inputFormat, const VPIStereoDisparityEstimatorCreationParams *params, VPIPayload *payload)
 Creates payload for vpiSubmitStereoDisparityEstimator. More...
 
VPIStatus vpiInitStereoDisparityEstimatorParams (VPIStereoDisparityEstimatorParams *params)
 Initializes VPIStereoDisparityEstimatorParams with default values. More...
 
VPIStatus vpiSubmitStereoDisparityEstimator (VPIStream stream, uint64_t backend, VPIPayload payload, VPIImage left, VPIImage right, VPIImage disparity, VPIImage confidenceMap, const VPIStereoDisparityEstimatorParams *params)
 Runs stereo processing on a pair of images and outputs a disparity map. More...
 

Detailed Description

Estimates disparity from a stereo pair.

Refer to Stereo Disparity Estimator for more details and usage examples.


Data Structure Documentation

◆ VPIStereoDisparityEstimatorCreationParams

struct VPIStereoDisparityEstimatorCreationParams

Structure that defines the parameters for vpiCreateStereoDisparityEstimator.

Definition at line 82 of file StereoDisparity.h.

+ Collaboration diagram for VPIStereoDisparityEstimatorCreationParams:
Data Fields
int32_t maxDisparity Maximum disparity for matching search.
  • Must be positive.
  • On OFA or OFA+PVA+VIC backend, maxDisparity must be 128 or 256.
  • On CUDA backends, maxDisparity must be >= 0 and <= 256.
int32_t downscaleFactor Output's downscale factor with respect to the input's resolution.
  • For OFA and OFA+PVA+VIC backend, the allowed values are 1, 2, 4 or 8.
  • For other backends, only 1 is supported, i.e. output dimension equals input's.
int8_t includeDiagonals Include diagonals or oblique paths in semi-global matching (SGM) computation.

Including diagonal paths makes the stereo disparity estimator slower, but increases the result quality. Including diagonal paths also increases memory usage in the CUDA backend. If zero means not to include diagonal paths, i.e. horizontal and vertical paths only. If not zero means to include diagonal paths, i.e. horizontal, vertical and oblique paths. It's only applicable when using CUDA and OFA backends.

◆ VPIStereoDisparityEstimatorParams

struct VPIStereoDisparityEstimatorParams

Structure that defines the parameters for vpiSubmitStereoDisparityEstimator.

Definition at line 204 of file StereoDisparity.h.

+ Collaboration diagram for VPIStereoDisparityEstimatorParams:
Data Fields
int32_t windowSize Represents the median filter size on OFA+PVA+VIC backend or census transform window size (other backends) used in the algorithm.
  • On CUDA backend this is ignored. A 9x7 window is used instead.
  • On OFA backend it is ignored.
  • On OFA+PVA+VIC backend, valid values are 1, 3, 5 or 7.
int32_t maxDisparity Maximum disparity for matching search.
  • Maximum disparity must be 0 (use from payload), or positive and less or equal to what's configured in payload.
  • For CUDA, it must be equal to the configured in the payload (or 0)
int32_t confidenceThreshold Confidence threshold above which disparity values are considered valid.

Only used in CUDA, and OFA+PVA+VIC backends.

  • Must be a value in U16 ranges, i.e. from 0 to 65535.
int32_t quality Ignored, do not use.
  • Must be a value between 1 and 8.
VPIStereoDisparityConfidenceType confidenceType Computation type to produce the confidence output.
int32_t minDisparity Minimum possible disparity value.

It's only applicable when using CUDA backend. Normally it is zero, but it may be adjusted depending on input images.

  • Must be between 0 and maximum disparity.
int32_t p1 Penalty on disparity changes of +/- 1 between neighbor pixels.

It is normally a small value, in (min, max) disparity range, used to penalize small changes in disparity, i.e. one. Using a lower P1 penalty allows for an adaptation to slanted or curver surfaces. (Please refer to paper_sgm on how penalties affect disparity values.) It's only applicable when using CUDA and OFA backends.

  • Must be a positive value.
  • Must be smaller than or equal to P2.
int32_t p2 Penalty on disparity changes of more than 1 between neighbor pixels.

It is normally a large value, in (min, max) disparity range, used to penalize large changes in disparity. Using a higher P2 penalty avoids abrupt disparity changes, i.e. discontinuities. (Please refer to paper_sgm on how penalties affect disparity values.) It's only applicable when using CUDA and OFA backends.

  • Must be a positive value.
  • Must be equal to or greater than P1.
  • in CUDA backend, must be less than 256.
  • In OFA backends, must be less than or equal to 217 - P1.
int32_t p2Alpha Alpha is used to enable adaptive large penalty (adaptive P2) feature.

It's used to better preserve object boundaries and thin objects. Higher alpha indicates lower adaptation of P2 in accordance to pixel intensities. It's the alpha value in: P2' = - (1 / alpha) * abs(I_cur - I_prev) + P2; where P2' is the adaptive version of the given P2 penalty. The abs(I_cur - I_prev) is the absolute difference in intensity between the current and previous pixel location along the direction of path cost evaluation. The intuition behind this adaptation is that large disparity differences should be penalized less if they coincide with large intensity changes. The alpha value can be 0, 1, 2, 4 or 8. Using an alpha value of 0 disables the adaptive P2 feature. It's only applicable when using OFA backend.

  • Must be 0, 1, 2, 4 or 8.
float uniqueness Uniqueness ratio, in [0, 1] range, is a margin by which best cost value should win over the second best.

A value of 1 means less pixels are marked invalid, as the full cost is considered when comparing against the current best cost. Values less than 1 mean only a ratio of the cost is considered for the best cost. A value of 0 means more pixels are marked invalid, as only disparity changes of +/- 1 pixels are considered. Use it to improve certainty of computed disparity. Use value outside [0, 1], such as -1, to disregard the uniqueness computation. It's only applicable when using CUDA backend.

int8_t numPasses Number of passes in memory-efficient semi-global matching (eSGM) computation.

A value of 1 or 2 means to do one forward or one forward and one backward passes per direction. A value of 3 means to do another forward pass after forward and backward passes. Setting number of passes to 3 makes the stereo disparity estimator slower, but increases the result quality. It's only applicable when using OFA backends.

  • Must be 1, 2 or 3.

Enumeration Type Documentation

◆ VPIStereoDisparityConfidenceType

#include </opt/nvidia/vpi3/include/vpi/algo/StereoDisparity.h>

Defines the way the confidence values are computed.

This computation needs the disparity difference from left to right (D_lr) and from right to left (D_rl), and potentially the maximum disparity (MAX_DISP). Lower confidence values (than the given confidence threshold parameter) make the output disparity invalid.

Enumerator
VPI_STEREO_CONFIDENCE_ABSOLUTE 

The U16 confidence value of a pixel is given by: [ 1 - abs(D_lr - D_rl) / MAX_DISP ] * 0xFFFF.

VPI_STEREO_CONFIDENCE_RELATIVE 

The U16 confidence value of a pixel is given by: [ 1 - abs(D_lr - D_rl) / D_lr ] * 0xFFFF.

Definition at line 187 of file StereoDisparity.h.

Function Documentation

◆ vpiInitStereoDisparityEstimatorCreationParams()

VPIStatus vpiInitStereoDisparityEstimatorCreationParams ( VPIStereoDisparityEstimatorCreationParams params)

#include </opt/nvidia/vpi3/include/vpi/algo/StereoDisparity.h>

Initializes VPIStereoDisparityEstimatorCreationParams with default values.

Defaults:

  • maxDisparity = 64
  • downscaleFactor = 1
  • includeDiagonals = 1
Parameters
[in]paramsStructure to be filled with default values.
Return values
VPI_ERROR_INVALID_ARGUMENTparams is NULL.
VPI_SUCCESSOperation executed successfully.

◆ vpiCreateStereoDisparityEstimator()

VPIStatus vpiCreateStereoDisparityEstimator ( uint64_t  backends,
int32_t  imageWidth,
int32_t  imageHeight,
VPIImageFormat  inputFormat,
const VPIStereoDisparityEstimatorCreationParams params,
VPIPayload payload 
)

#include </opt/nvidia/vpi3/include/vpi/algo/StereoDisparity.h>

Creates payload for vpiSubmitStereoDisparityEstimator.

Parameters
[in]backendsVPI backends that are eligible to execute the algorithm.
[in]imageWidth,imageHeightInput image dimensions.
  • Must be >= 0.
  • On OFA, input width and height must be between 16 and 16384 after applying downscaleFactor.
  • On OFA+PVA+VIC, input width must be at least max(64, maxDisparity/downscaleFactor) * downscaleFactor.
[in]inputFormatInput image format.
[in]paramsCreation parameters. Pass NULL to use defaults given by vpiInitStereoDisparityEstimatorCreationParams.
[out]payloadPointer to the payload variable that receives the created handle.
Return values
VPI_ERROR_INVALID_IMAGE_FORMATinputFormat is not supported.
VPI_ERROR_INVALID_ARGUMENTpayload is NULL.
VPI_ERROR_INVALID_ARGUMENTimageWidth or imageHeight outside valid range.
VPI_ERROR_INVALID_ARGUMENTbackends refers to an invalid backend.
VPI_ERROR_INVALID_ARGUMENTMaximum disparity in params outside valid range.
VPI_ERROR_NOT_IMPLEMENTEDStereo Disparity Estimator algorithm is not supported by given backends.
VPI_ERROR_INVALID_OPERATIONBackend hardware not available.
VPI_ERROR_INVALID_CONTEXTCurrent context is destroyed.
VPI_ERROR_OUT_OF_MEMORYCannot allocate required resources.
VPI_ERROR_INVALID_OPERATIONBackend isn't enabled in current context.
VPI_SUCCESSOperation executed successfully.

◆ vpiInitStereoDisparityEstimatorParams()

VPIStatus vpiInitStereoDisparityEstimatorParams ( VPIStereoDisparityEstimatorParams params)

#include </opt/nvidia/vpi3/include/vpi/algo/StereoDisparity.h>

Initializes VPIStereoDisparityEstimatorParams with default values.

Defaults:

  • windowSize = 5
  • maxDisparity = 0 (uses disparity set during payload construction)
  • confidenceThreshold = 32767
  • confidenceType = VPI_STEREO_CONFIDENCE_ABSOLUTE
  • minDisparity = 0
  • p1 = 3
  • p2 = 48
  • p2Alpha = 0
  • uniqueness = -1
  • numPasses = 3
Parameters
[in]paramsStructure to be filled with default values.
Return values
VPI_ERROR_INVALID_ARGUMENTparams is NULL.
VPI_SUCCESSOperation executed successfully.

◆ vpiSubmitStereoDisparityEstimator()

VPIStatus vpiSubmitStereoDisparityEstimator ( VPIStream  stream,
uint64_t  backend,
VPIPayload  payload,
VPIImage  left,
VPIImage  right,
VPIImage  disparity,
VPIImage  confidenceMap,
const VPIStereoDisparityEstimatorParams params 
)

#include </opt/nvidia/vpi3/include/vpi/algo/StereoDisparity.h>

Runs stereo processing on a pair of images and outputs a disparity map.

Parameters
[in]streamThe stream where the operation will be queued in.
  • Must not be NULL.
  • Stream must have enabled the backends that will execute the algorithm.
[in]backendBackend that will execute the algorithm.
  • Must be the backend specified during payload creation or 0 as a shorthand to use this backend.
[in]payloadPayload to be submitted along the other parameters.
[in]leftLeft stereo input image.
  • Must not be NULL.
  • Must have same format and dimensions as the one specified during payload creation.
  • Image must have enabled the backends that will execute the algorithm.
[in]rightRight stereo input image.
  • Must not be NULL.
  • Must have same format and dimensions as left.
  • Image must have enabled the backends that will execute the algorithm.
[out]disparityImage where the disparity values will be written to. Returned values are in Q10.5 format, i.e., signed fixed point with 5 fractional bits. Divide it by 32.0f to convert it to floating point.
[out]confidenceMapImage containing a confidence score telling how accurate the disparity is. Pass NULL if it isn't needed. Returned confidence range from 0 to 65535. The higher the confidence, more accurate is the corresponding disparity.
  • OFA backend doesn't support confidence map output, it must be NULL.
  • Image format must be VPI_IMAGE_FORMAT_U16
  • Must be same dimensions as disparity.
  • Image must have enabled the backends that will execute the algorithm.
[in]paramsPointer to algorithm control parameters. If NULL, it'll use the defaults given by vpiInitStereoDisparityEstimatorParams.
Return values
VPI_ERROR_INVALID_ARGUMENTstream is NULL.
VPI_ERROR_INVALID_ARGUMENTleft, right or disparity are NULL.
VPI_ERROR_INVALID_ARGUMENTMaximum disparity in params outside valid range.
VPI_ERROR_INVALID_ARGUMENTWindow size in params outside valid range.
VPI_ERROR_INVALID_ARGUMENTleft and right images must have the same dimensions.
VPI_ERROR_INVALID_ARGUMENTInput image dimensions must be equal to what's configured in payload.
VPI_ERROR_INVALID_ARGUMENTconfidenceMap and disparity must have same dimensions.
VPI_ERROR_INVALID_ARGUMENTBackend doesn't output confidence map.
VPI_ERROR_INVALID_IMAGE_FORMATleft and right format don't match payload image format.
VPI_ERROR_INVALID_IMAGE_FORMATdisparity format is invalid.
VPI_ERROR_INVALID_PAYLOAD_TYPEpayload is invalid.
VPI_ERROR_INVALID_OPERATIONThe needed backends aren't enabled in stream, left, right, disparity or confidenceMap.
VPI_SUCCESSOperation executed successfully.