NVIDIA® Vision Programming Interface (VPI) is a software library that provides Computer Vision / Image Processing algorithms implemented on several computing hardware platforms available in NVIDIA embedded or discrete devices.
VPI provides seamless access to a variety of computing hardware that must be accessed through different and sometimes incompatible APIs, such as OpenCV and CUDA SDK, or when public APIs do not exist yet, such as PVA (Programmable Vision Accelerator) and VIC (Video and Image Compositor).
Within VPI, the same algorithm is implemented in different backends, such as CPU, GPU, PVA1 and VIC2. The processing pipeline can be setup to fully utilize the installed computing capacity of the target device. For example, a Deep Learning inference stage can be run for one frame on the GPU while PVA and VIC are running pre-processing stages for the next frame. At the same time, the CPU is free to perform GUI tasks such as showing the current result back to the user, or executing post-processing tasks on the previous frame.
To achieve high throughput, VPI provides seamless shared, zero-copy, memory mapping among the different backends it supports, depending on the memory characteristics (dimensions, element type, alignment, etc). The API is designed so that memory allocations can occur in an initialization stage, usually run once. Then, during the main loop of the application, the algorithms reuses the pre-allocated memory, as needed, avoiding memory fragmentation and enabling usage in contexts that need to conform to stricter memory allocation characteristics, such as automotive software3.
Some examples of algorithms provided by VPI:
|
|
|
|
|
|
1 PVA backend is only available on Jetson Xavier devices, such as Jetson AGX Xavier and Jetson Xavier NX.
2 VIC backend is only available on Jetson devices.
3 Although the API is designed with safety in mind, VPI library isn't itself safety-certified.