VPI - Vision Programming Interface

0.1.0 Release

Algorithms

The algorithms supported by VPI are described in the following sections. An overview of the algorithm, instructions on how to use the algorithm, a list of any backend limitations, and the API reference are provided.

AlgorithmBackend Support
CUDAPVACPU
Stereo Disparity Estimator yes yes yes
KLT Bounding Box Tracker yes yes yes
Gaussian Pyramid Generator yes yes yes
Image Convolver yes yes yes
Separable Image Convolver yes yes yes
Box Image Filter yes yes yes
Gaussian Image Filter yes yes yes
Bilateral Image Filter yes no yes
Image Resampler yes no yes
Harris Keypoint Detector yes no yes
vpiSubmitGaussianPyramidGenerator
VPIStatus vpiSubmitGaussianPyramidGenerator(VPIStream stream, VPIImage input, VPIPyramid output)
Computes the Gaussian pyramid from the input image.
vpiArrayCreate
VPIStatus vpiArrayCreate(uint32_t capacity, VPIArrayType fmt, uint32_t flags, VPIArray *array)
Create an empty array instance with the specified flags.
vpiPyramidCreate
VPIStatus vpiPyramidCreate(uint32_t width, uint32_t height, VPIImageType fmt, uint32_t numLevels, float scale, uint32_t flags, VPIPyramid *pyr)
Create an empty image pyramid instance with the specified flags.
ImageConvolver.h
vpiCreateStereoDisparityEstimator
VPIStatus vpiCreateStereoDisparityEstimator(VPIStream stream, uint32_t imageWidth, uint32_t imageHeight, const VPIImageType inputType, const uint32_t maxDisparity, VPIPayload *payload)
Creates payload for vpiSubmitStereoDisparityEstimator.
VPIImageType
VPIImageType
Image formats.
Definition: Types.h:172
VPIHarrisKeypointDetectorParams::sensitivity
float sensitivity
Specifies sensitivity threshold from the Harris-Stephens equation.
Definition: HarrisKeypointDetector.h:92
VPIKLTTrackedBoundingBox
Stores a bounding box that is being tracked by KLT Tracker.
Definition: Types.h:354
VPIKLTTrackedBoundingBox::bbox
VPIBoundingBox bbox
Bounding box being tracked.
Definition: Types.h:357
vpiSubmitBilateralImageFilter
VPIStatus vpiSubmitBilateralImageFilter(VPIStream stream, VPIImage input, VPIImage output, uint32_t kernelSize, float sigmaRange, float sigmaSpace, VPIBoundaryCond boundary)
Runs a 2D bilateral filter over an image.
vpiSubmitBoxImageFilter
VPIStatus vpiSubmitBoxImageFilter(VPIStream stream, VPIImage input, VPIImage output, uint32_t kernelSizeX, uint32_t kernelSizeY, VPIBoundaryCond boundary)
Runs a 2D box filter over an image.
BoxImageFilter.h
GaussianImageFilter.h
KLTBoundingBoxTracker.h
VPIKLTTrackedBoundingBox::trackingStatus
uint8_t trackingStatus
Tracking status of this bounding box.
Definition: Types.h:364
VPIHomographyTransform2D
Stores a generic 2D homography transform.
Definition: Types.h:326
vpiCreateHarrisKeypointDetector
VPIStatus vpiCreateHarrisKeypointDetector(VPIStream stream, uint32_t inputWidth, uint32_t inputHeight, VPIPayload *payload)
Creates a Harris Keypoint Detector payload.
VPIArrayData::type
VPIArrayType type
Type of each array element.
Definition: Array.h:118
vpiArrayUnlock
VPIStatus vpiArrayUnlock(VPIArray array)
Releases the lock on array object.
VPIHarrisKeypointDetectorParams
Structure that defines the parameters for vpiSubmitHarrisKeypointDetector.
Definition: HarrisKeypointDetector.h:80
vpiStreamSync
VPIStatus vpiStreamSync(VPIStream stream)
Blocks the calling thread until all submitted commands in this stream queue are done (queue is empty)...
VPIKLTBoundingBoxTrackerParams::maxScaleChange
float maxScaleChange
Maximum relative scale change.
Definition: KLTBoundingBoxTracker.h:113
vpiSubmitImageResampler
VPIStatus vpiSubmitImageResampler(VPIStream stream, VPIImage input, VPIImage output, VPIInterpolationType interpolationType, VPIBoundaryCond boundary)
Runs a generic resampling algorithm on a 2D image.
HarrisKeypointDetector.h
VPIHarrisKeypointDetectorParams::strengthThresh
float strengthThresh
Specifies the minimum threshold with which to eliminate Harris Corner scores.
Definition: HarrisKeypointDetector.h:89
vpiSubmitStereoDisparityEstimator
VPIStatus vpiSubmitStereoDisparityEstimator(VPIPayload payload, VPIImage left, VPIImage right, VPIImage disparity, const VPIStereoDisparityEstimatorParams *params)
Runs stereo processing on a pair of images, outputs a disparity map.
StereoDisparityEstimator.h
VPIHarrisKeypointDetectorParams::gradientSize
uint32_t gradientSize
Gradient window size.
Definition: HarrisKeypointDetector.h:83
VPIArrayData::size
uint32_t size
Number of elements in the array.
Definition: Array.h:119
vpiArrayWrapHostMem
VPIStatus vpiArrayWrapHostMem(const VPIArrayData *arrayData, uint32_t flags, VPIArray *array)
Create an array object by wrapping around an existing host-memory block.
VPIKLTBoundingBoxTrackerParams::trackingType
VPIKLTBoundingBoxTrackerType trackingType
Type of KLT tracking that will be performed.
Definition: KLTBoundingBoxTracker.h:120
VPI_ARRAY_TYPE_HOMOGRAPHY_TRANSFORM_2D
VPIHomographyTransform2D element.
Definition: Types.h:217
vpiSubmitHarrisKeypointDetector
VPIStatus vpiSubmitHarrisKeypointDetector(VPIPayload payload, VPIImage input, VPIArray outFeatures, VPIArray outScores, const VPIHarrisKeypointDetectorParams *params)
Submits Harris Keypoint Detector operation to the stream associated with the payload.
vpiSubmitKLTBoundingBoxTracker
VPIStatus vpiSubmitKLTBoundingBoxTracker(VPIPayload payload, VPIImage templateImage, VPIArray inputBoxList, VPIArray inputPredictionList, VPIImage referenceImage, VPIArray outputBoxList, VPIArray outputEstimationList, const VPIKLTBoundingBoxTrackerParams *params)
Runs KLT Tracker on two frames.
vpiContextDestroy
void vpiContextDestroy(VPIContext ctx)
Destroy a context instance as well as all resources it owns.
VPIArrayData::capacity
uint32_t capacity
Maximum number of elements that the array can hold.
Definition: Array.h:120
VPI_LOCK_READ
Lock memory only for reading.
Definition: Types.h:386
VPIArrayData::data
void * data
Points to the first element of the array.
Definition: Array.h:122
GaussianPyramidGenerator.h
SeparableImageConvolver.h
vpiSubmitImageConvolver
VPIStatus vpiSubmitImageConvolver(VPIStream stream, VPIImage input, VPIImage output, const float *kernelData, uint32_t kernelWidth, uint32_t kernelHeight, VPIBoundaryCond boundary)
Runs a generic 2D convolution over an image.
VPIHarrisKeypointDetectorParams::minNMSDistance
float minNMSDistance
Non-maximum suppression radius, set to 0 to disable it.
Definition: HarrisKeypointDetector.h:95
VPIHomographyTransform2D::mat3
float mat3[3][3]
3x3 homogeneous matrix that defines the homography.
Definition: Types.h:328
VPIKLTBoundingBoxTrackerParams::maxTranslationChange
float maxTranslationChange
Maximum relative translation change.
Definition: KLTBoundingBoxTracker.h:117
ImageResampler.h
VPIKLTBoundingBoxTrackerParams::nccThresholdStop
float nccThresholdStop
Threshold to stop estimating.
Definition: KLTBoundingBoxTracker.h:109
VPIKLTTrackedBoundingBox::templateStatus
uint8_t templateStatus
Status of the template related to this bounding box.
Definition: Types.h:371
VPI_ARRAY_TYPE_KEYPOINT
VPIKeypoint element.
Definition: Types.h:216
VPIKLTBoundingBoxTrackerParams::nccThresholdKill
float nccThresholdKill
Threshold to consider template tracking was lost.
Definition: KLTBoundingBoxTracker.h:108
vpiCreateKLTBoundingBoxTracker
VPIStatus vpiCreateKLTBoundingBoxTracker(VPIStream stream, uint32_t imageWidth, uint32_t imageHeight, VPIImageType imageType, VPIPayload *payload)
Creates payload for vpiSubmitKLTBoundingBoxTracker.
VPI_ARRAY_TYPE_KLT_TRACKED_BOUNDING_BOX
VPIKLTTrackedBoundingBox element.
Definition: Types.h:218
VPIKLTBoundingBoxTrackerParams
Structure that defines the parameters for vpiCreateKLTBoundingBoxTracker.
Definition: KLTBoundingBoxTracker.h:104
VPI_IMAGE_TYPE_Y16
unsigned 16-bit grayscale/luma.
Definition: Types.h:176
VPIStereoDisparityEstimatorParams::windowSize
uint32_t windowSize
width of Census Transform window for disparity features.
Definition: StereoDisparityEstimator.h:82
VPIImage
struct VPIImageImpl * VPIImage
Definition: Types.h:153
VPIKLTBoundingBoxTrackerParams::nccThresholdUpdate
float nccThresholdUpdate
Threshold for requiring template update.
Definition: KLTBoundingBoxTracker.h:107
vpiImageGetSize
VPIStatus vpiImageGetSize(VPIImage img, uint32_t *width, uint32_t *height)
Get the image size in pixels.
vpiImageGetType
VPIStatus vpiImageGetType(VPIImage img, VPIImageType *type)
Get the image type.
VPIArrayData
Stores information about array characteristics and content.
Definition: Array.h:116
VPIHarrisKeypointDetectorParams::blockSize
uint32_t blockSize
Block window size used to compute the Harris Corner score.
Definition: HarrisKeypointDetector.h:86
vpiImageCreate
VPIStatus vpiImageCreate(uint32_t width, uint32_t height, VPIImageType type, uint32_t flags, VPIImage *img)
Create an empty image instance with the specified flags.
VPI_BOUNDARY_COND_ZERO
All pixels outside the image are considered to be zero.
Definition: Types.h:204
vpiArrayLock
VPIStatus vpiArrayLock(VPIArray array, VPILockMode mode, VPIArrayData *arrayData)
Acquires the lock on array object and returns a pointer to array data.
vpiSubmitSeparableImageConvolver
VPIStatus vpiSubmitSeparableImageConvolver(VPIStream stream, VPIImage input, VPIImage output, const float *kernelXData, uint32_t kernelXSize, const float *kernelYData, uint32_t kernelYSize, VPIBoundaryCond boundary)
Runs a generic 2D convolution over an image.
VPIStereoDisparityEstimatorParams::maxDisparity
uint32_t maxDisparity
Maximum disparity for matching search.
Definition: StereoDisparityEstimator.h:83
VPIStereoDisparityEstimatorParams
Structure that defines the parameters for vpiCreateStereoDisparityEstimator.
Definition: StereoDisparityEstimator.h:80
VPIPayload
struct VPIPayloadImpl * VPIPayload
A handle to an algorithm payload.
Definition: Types.h:164
VPIArray
struct VPIArrayImpl * VPIArray
Definition: Types.h:129
vpiPayloadDestroy
void vpiPayloadDestroy(VPIPayload payload)
Deallocates the payload object and all associated resources.
vpiArrayInvalidate
VPIStatus vpiArrayInvalidate(VPIArray array)
This method is useful for unmanaged arrays only (created with 'vpiArrayWrap*`).
VPI_KLT_INVERSE_COMPOSITIONAL
Inverse compositional algorithm for KLT tracker.
Definition: KLTBoundingBoxTracker.h:91
VPIKLTBoundingBoxTrackerParams::numberOfIterationsScaling
uint32_t numberOfIterationsScaling
Number of Inverse compositional iterations of scale estimations.
Definition: KLTBoundingBoxTracker.h:106
BilateralImageFilter.h
VPI_ARRAY_TYPE_U32
unsigned 32-bit.
Definition: Types.h:215
VPIStream
struct VPIStreamImpl * VPIStream
Definition: Types.h:147
VPIPyramid
struct VPIPyramidImpl * VPIPyramid
Definition: Types.h:159
vpiSubmitGaussianImageFilter
VPIStatus vpiSubmitGaussianImageFilter(VPIStream stream, VPIImage input, VPIImage output, uint32_t kernelSizeX, uint32_t kernelSizeY, float sigmaX, float sigmaY, VPIBoundaryCond boundary)
Runs a 2D Gaussian filter over an image.
VPIBoundingBox
Stores a generic 2D bounding box.
Definition: Types.h:344
VPI_INTERP_LINEAR_FAST
Fast linear interpolation.
Definition: Types.h:258