Stereo Disparity Estimator

Estimates disparity from a stereo pair. More...

Data Structures
struct	VPIStereoDisparityEstimatorCreationParams
	Structure that defines the parameters for vpiCreateStereoDisparityEstimator. More...
struct	VPIStereoDisparityEstimatorParams
	Structure that defines the parameters for vpiSubmitStereoDisparityEstimator. More...

Enumerations
enum	VPIStereoDisparityConfidenceType
	Defines the way the confidence values are computed. More...

Functions
VPIStatus	vpiInitStereoDisparityEstimatorCreationParams (VPIStereoDisparityEstimatorCreationParams *params)
	Initializes VPIStereoDisparityEstimatorCreationParams with default values. More...
VPIStatus	vpiCreateStereoDisparityEstimator (uint64_t backends, int32_t imageWidth, int32_t imageHeight, VPIImageFormat inputFormat, const VPIStereoDisparityEstimatorCreationParams *params, VPIPayload *payload)
	Creates payload for vpiSubmitStereoDisparityEstimator. More...
VPIStatus	vpiInitStereoDisparityEstimatorParams (VPIStereoDisparityEstimatorParams *params)
	Initializes VPIStereoDisparityEstimatorParams with default values. More...
VPIStatus	vpiSubmitStereoDisparityEstimator (VPIStream stream, uint64_t backend, VPIPayload payload, VPIImage left, VPIImage right, VPIImage disparity, VPIImage confidenceMap, const VPIStereoDisparityEstimatorParams *params)
	Runs stereo processing on a pair of images and outputs a disparity map. More...

Detailed Description

Estimates disparity from a stereo pair.

Refer to Stereo Disparity Estimator for more details and usage examples.

---

Data Structure Documentation

VPIStereoDisparityEstimatorCreationParams

struct VPIStereoDisparityEstimatorCreationParams

Structure that defines the parameters for vpiCreateStereoDisparityEstimator.

Definition at line 82 of file StereoDisparity.h.

Collaboration diagram for VPIStereoDisparityEstimatorCreationParams:

Data Fields
int32_t	maxDisparity	Maximum disparity for matching search. Must be positive. On CPU backend, maxDisparity must be <= 64. On PVA backend, maxDisparity must be == 64. On OFA or OFA+PVA+VIC backend, maxDisparity must be 128 or 256. On CUDA backends, maxDisparity must be >= 0 and <= 256. On PVA+NVENC+VIC backends, range from >= 0 and <= 256 is accepted, but internally this value is ignored and 256 is used instead.
int32_t	downscaleFactor	Output's downscale factor with respect to the input's resolution. For PVA+NVENC+VIC backend, only 4 is supported. Setting it to 4 leads to faster execution speeds as NVENC engine internal output has 1/4th resolution. For OFA and OFA+PVA+VIC backend, the allowed values are 1, 2, 4 or 8. For other backends, only 1 is supported, i.e. output dimension equals input's.
int8_t	includeDiagonals	Include diagonals or oblique paths in semi-global matching (SGM) computation. Including diagonal paths makes the stereo disparity estimator slower, but increases the result quality. Including diagonal paths also increases memory usage in the CUDA backend. If zero means not to include diagonal paths, i.e. horizontal and vertical paths only. If not zero means to include diagonal paths, i.e. horizontal, vertical and oblique paths. It's only applicable when using CUDA and OFA backends.

VPIStereoDisparityEstimatorParams

struct VPIStereoDisparityEstimatorParams

Structure that defines the parameters for vpiSubmitStereoDisparityEstimator.

Definition at line 220 of file StereoDisparity.h.

Collaboration diagram for VPIStereoDisparityEstimatorParams:

Data Fields
int32_t	windowSize	Represents the median filter size (on PVA+NVENC+VIC or OFA+PVA+VIC backend) or census transform window size (other backends) used in the algorithm. On PVA backend, it must be 5. On CPU backend, it must be >= 1. On CUDA backend this is ignored. A 9x7 window is used instead. On OFA backend it is ignored. On OFA+PVA+VIC backend, valid values are 1, 3, 5 or 7. On PVA+NVENC+VIC backend, valid values are 1, 3, 5 or 7.
int32_t	maxDisparity	Maximum disparity for matching search. Maximum disparity must be 0 (use from payload), or positive and less or equal to what's configured in payload. For CUDA, it must be equal to the configured in the payload (or 0)
int32_t	confidenceThreshold	Confidence threshold above which disparity values are considered valid. Only used in CUDA, PVA+NVENC+VIC, and OFA+PVA+VIC backends. Must be a value in U16 ranges, i.e. from 0 to 65535.
int32_t	quality	Quality of disparity output. It's only applicable when using PVA+NVENC+VIC backend. The higher the value, better the quality and possibly slower perf. Must be a value between 1 and 8.
VPIStereoDisparityConfidenceType	confidenceType	Computation type to produce the confidence output. Use VPI_STEREO_CONFIDENCE_ABSOLUTE for absolute difference between disparities computation. Use VPI_STEREO_CONFIDENCE_RELATIVE for relative difference between disparities computation.
int32_t	minDisparity	Minimum possible disparity value. It's only applicable when using CUDA backend. Normally it is zero, but it may be adjusted depending on input images. Must be between 0 and maximum disparity.
int32_t	p1	Penalty on disparity changes of +/- 1 between neighbor pixels. It is normally a small value, in (min, max) disparity range, used to penalize small changes in disparity, i.e. one. Using a lower P1 penalty allows for an adaptation to slanted or curver surfaces. (Please refer to paper_sgm on how penalties affect disparity values.) It's only applicable when using CUDA and OFA backends. Must be a positive value. Must be smaller than or equal to P2.
int32_t	p2	Penalty on disparity changes of more than 1 between neighbor pixels. It is normally a large value, in (min, max) disparity range, used to penalize large changes in disparity. Using a higher P2 penalty avoids abrupt disparity changes, i.e. discontinuities. (Please refer to paper_sgm on how penalties affect disparity values.) It's only applicable when using CUDA and OFA backends. Must be a positive value. Must be equal to or greater than P1. in CUDA backend, must be less than 256. In OFA backends, must be less than or equal to 217 - P1.
int32_t	p2Alpha	Alpha is used to enable adaptive large penalty (adaptive P2) feature. It's used to better preserve object boundaries and thin objects. Higher alpha indicates lower adaptation of P2 in accordance to pixel intensities. It's the alpha value in: P2' = - (1 / alpha) * abs(I_cur - I_prev) + P2; where P2' is the adaptive version of the given P2 penalty. The abs(I_cur - I_prev) is the absolute difference in intensity between the current and previous pixel location along the direction of path cost evaluation. The intuition behind this adaptation is that large disparity differences should be penalized less if they coincide with large intensity changes. The alpha value can be 0, 1, 2, 4 or 8. Using an alpha value of 0 disables the adaptive P2 feature. It's only applicable when using OFA backend. Must be 0, 1, 2, 4 or 8.
float	uniqueness	Uniqueness ratio, in [0, 1] range, is a margin by which best cost value should win over the second best. A value of 1 means less pixels are marked invalid, as the full cost is considered when comparing against the current best cost. Values less than 1 mean only a ratio of the cost is considered for the best cost. A value of 0 means more pixels are marked invalid, as only disparity changes of +/- 1 pixels are considered. Use it to improve certainty of computed disparity. Use value outside [0, 1], such as -1, to disregard the uniqueness computation. It's only applicable when using CUDA backend.
int8_t	numPasses	Number of passes in memory-efficient semi-global matching (eSGM) computation. A value of 1 or 2 means to do one forward or one forward and one backward passes per direction. A value of 3 means to do another forward pass after forward and backward passes. Setting number of passes to 3 makes the stereo disparity estimator slower, but increases the result quality. It's only applicable when using OFA backends. Must be 1, 2 or 3.

Enumeration Type Documentation

VPIStereoDisparityConfidenceType

enum VPIStereoDisparityConfidenceType

#include </opt/nvidia/vpi2/include/vpi/algo/StereoDisparity.h>

Defines the way the confidence values are computed.

This computation needs the disparity difference from left to right (D_lr) and from right to left (D_rl), and potentially the maximum disparity (MAX_DISP). Lower confidence values (than the given confidence threshold parameter) make the output disparity invalid.

Enumerator
VPI_STEREO_CONFIDENCE_ABSOLUTE	The U16 confidence value of a pixel is given by: [ 1 - abs(D_lr - D_rl) / MAX_DISP ] * 0xFFFF.
VPI_STEREO_CONFIDENCE_RELATIVE	The U16 confidence value of a pixel is given by: [ 1 - abs(D_lr - D_rl) / D_lr ] * 0xFFFF.

Definition at line 203 of file StereoDisparity.h.

Function Documentation

vpiInitStereoDisparityEstimatorCreationParams()

VPIStatus vpiInitStereoDisparityEstimatorCreationParams

(

VPIStereoDisparityEstimatorCreationParams *

params

)

#include </opt/nvidia/vpi2/include/vpi/algo/StereoDisparity.h>

Initializes VPIStereoDisparityEstimatorCreationParams with default values.

Defaults:

• maxDisparity = 64
• downscaleFactor = 1
• includeDiagonals = 1

Parameters

[in]

params

Structure to be filled with default values.

Return values

VPI_ERROR_INVALID_ARGUMENT	params is NULL.
VPI_SUCCESS	Operation executed successfully.

vpiCreateStereoDisparityEstimator()

VPIStatus vpiCreateStereoDisparityEstimator	(	uint64_t	backends,
		int32_t	imageWidth,
		int32_t	imageHeight,
		VPIImageFormat	inputFormat,
		const VPIStereoDisparityEstimatorCreationParams *	params,
		VPIPayload *	payload
	)

#include </opt/nvidia/vpi2/include/vpi/algo/StereoDisparity.h>

Creates payload for vpiSubmitStereoDisparityEstimator.

Parameters

[in]	backends	VPI backends that are eligible to execute the algorithm. Valid values: VPI_BACKEND_CPU VPI_BACKEND_CUDA VPI_BACKEND_OFA VPI_BACKEND_PVA VPI_BACKEND_PVA | VPI_BACKEND_OFA | VPI_BACKEND_VIC VPI_BACKEND_PVA | VPI_BACKEND_NVENC | VPI_BACKEND_VIC Backends must be enabled in current context.
[in]	imageWidth,imageHeight	Input image dimensions. Must be >= 0. On PVA, input dimensions must be 480x270. On PVA+NVENC+VIC, input dimensions must be 1920x1080. On OFA, input width and height must be between 16 and 16384 after applying downscaleFactor. On OFA+PVA+VIC, input width must be at least max(64, maxDisparity/downscaleFactor) * downscaleFactor.
[in]	inputFormat	Input image format. Supported image formats are: Formats CUDA CPU PVA OFA OFA+PVA+VIC PVA+NVENC+VIC VPI_IMAGE_FORMAT_U8 * * VPI_IMAGE_FORMAT_U16 * * * VPI_IMAGE_FORMAT_Y8 * * VPI_IMAGE_FORMAT_Y8_BL * * VPI_IMAGE_FORMAT_Y8_ER * * VPI_IMAGE_FORMAT_Y8_ER_BL * * VPI_IMAGE_FORMAT_Y16 * * VPI_IMAGE_FORMAT_Y16_BL * * VPI_IMAGE_FORMAT_Y16_ER * * * VPI_IMAGE_FORMAT_Y16_ER_BL * * * VPI_IMAGE_FORMAT_NV12 * * VPI_IMAGE_FORMAT_NV12_BL * * VPI_IMAGE_FORMAT_NV12_ER * * VPI_IMAGE_FORMAT_NV12_ER_BL * * VPI_IMAGE_FORMAT_NV24 * * VPI_IMAGE_FORMAT_NV24_BL * * VPI_IMAGE_FORMAT_NV24_ER * * VPI_IMAGE_FORMAT_NV24_ER_BL * *
[in]	params	Creation parameters. Pass NULL to use defaults given by vpiInitStereoDisparityEstimatorCreationParams.
[out]	payload	Pointer to the payload variable that receives the created handle.

Return values

VPI_ERROR_INVALID_IMAGE_FORMAT	inputFormat is not supported.
VPI_ERROR_INVALID_ARGUMENT	payload is NULL.
VPI_ERROR_INVALID_ARGUMENT	imageWidth or imageHeight outside valid range.
VPI_ERROR_INVALID_ARGUMENT	backends refers to an invalid backend.
VPI_ERROR_INVALID_ARGUMENT	Maximum disparity in params outside valid range.
VPI_ERROR_NOT_IMPLEMENTED	Stereo Disparity Estimator algorithm is not supported by given backends.
VPI_ERROR_INVALID_OPERATION	Backend hardware not available.
VPI_ERROR_INVALID_CONTEXT	Current context is destroyed.
VPI_ERROR_OUT_OF_MEMORY	Cannot allocate required resources.
VPI_ERROR_INVALID_OPERATION	Backend isn't enabled in current context.
VPI_SUCCESS	Operation executed successfully.

vpiInitStereoDisparityEstimatorParams()

VPIStatus vpiInitStereoDisparityEstimatorParams

(

VPIStereoDisparityEstimatorParams *

params

)

#include </opt/nvidia/vpi2/include/vpi/algo/StereoDisparity.h>

Initializes VPIStereoDisparityEstimatorParams with default values.

Defaults:

• windowSize = 5
• maxDisparity = 0 (uses disparity set during payload construction)
• confidenceThreshold = 32767
• quality = 6
• confidenceType = VPI_STEREO_CONFIDENCE_ABSOLUTE
• minDisparity = 0
• p1 = 3
• p2 = 48
• p2Alpha = 0
• uniqueness = -1
• numPasses = 3

Parameters

[in]

params

Structure to be filled with default values.

Return values

VPI_ERROR_INVALID_ARGUMENT	params is NULL.
VPI_SUCCESS	Operation executed successfully.

vpiSubmitStereoDisparityEstimator()

VPIStatus vpiSubmitStereoDisparityEstimator	(	VPIStream	stream,
		uint64_t	backend,
		VPIPayload	payload,
		VPIImage	left,
		VPIImage	right,
		VPIImage	disparity,
		VPIImage	confidenceMap,
		const VPIStereoDisparityEstimatorParams *	params
	)

#include </opt/nvidia/vpi2/include/vpi/algo/StereoDisparity.h>

Runs stereo processing on a pair of images and outputs a disparity map.

Parameters

[in]	stream	The stream where the operation will be queued in. Must not be NULL. Stream must have enabled the backends that will execute the algorithm.
[in]	backend	Backend that will execute the algorithm. Must be the backend specified during payload creation or 0 as a shorthand to use this backend.
[in]	payload	Payload to be submitted along the other parameters.
[in]	left	Left stereo input image. Must not be NULL. Must have same format and dimensions as the one specified during payload creation. Image must have enabled the backends that will execute the algorithm.
[in]	right	Right stereo input image. Must not be NULL. Must have same format and dimensions as left. Image must have enabled the backends that will execute the algorithm.
[out]	disparity	Image where the disparity values will be written to. Returned values are in Q10.5 format, i.e., signed fixed point with 5 fractional bits. Divide it by 32.0f to convert it to floating point. Must not be NULL. On OFA backend, image format must be VPI_IMAGE_FORMAT_S16_BL On other backends, image format must be VPI_IMAGE_FORMAT_S16. On PVA+NVENC+VIC backend, image format can be VPI_IMAGE_FORMAT_U16. On OFA+PVA+VIC backend, and if windowSize==1, confidenceMap==NULL and confidenceThreshold==0, the image format can be either VPI_IMAGE_FORMAT_S16 or VPI_IMAGE_FORMAT_S16_BL. Dimensions must be equal to input, but downscaled by VPIStereoDisparityEstimatorCreationParams::downscaleFactor passed in params. Image must have enabled the backends that will execute the algorithm.
[out]	confidenceMap	Image containing a confidence score telling how accurate the disparity is. Pass NULL if it isn't needed. Returned confidence range from 0 to 65535. The higher the confidence, more accurate is the corresponding disparity. CPU, PVA and OFA backends don't support confidence map output, it must be NULL. If NULL on PVA+NVENC+VIC backend, the confidenceThreshold parameter will be ignored. Image format must be VPI_IMAGE_FORMAT_U16 Must be same dimensions as disparity. Image must have enabled the backends that will execute the algorithm.
[in]	params	Pointer to algorithm control parameters. If NULL, it'll use the defaults given by vpiInitStereoDisparityEstimatorParams.

Return values

VPI_ERROR_INVALID_ARGUMENT	stream is NULL.
VPI_ERROR_INVALID_ARGUMENT	left, right or disparity are NULL.
VPI_ERROR_INVALID_ARGUMENT	Maximum disparity in params outside valid range.
VPI_ERROR_INVALID_ARGUMENT	Window size in params outside valid range.
VPI_ERROR_INVALID_ARGUMENT	left and right images must have the same dimensions.
VPI_ERROR_INVALID_ARGUMENT	Input image dimensions must be equal to what's configured in payload.
VPI_ERROR_INVALID_ARGUMENT	confidenceMap and disparity must have same dimensions.
VPI_ERROR_INVALID_ARGUMENT	Backend doesn't output confidence map.
VPI_ERROR_INVALID_IMAGE_FORMAT	left and right format don't match payload image format.
VPI_ERROR_INVALID_IMAGE_FORMAT	disparity format is invalid.
VPI_ERROR_INVALID_PAYLOAD_TYPE	payload is invalid.
VPI_ERROR_INVALID_OPERATION	The needed backends aren't enabled in stream, left, right, disparity or confidenceMap.
VPI_SUCCESS	Operation executed successfully.