VPI - Vision Programming Interface

3.1 Release


The C API provides low level access to all VPI functionality. It gives the developer more control on resource management, pipeline construction leading to more efficient processing in time and memory consumption.


  • Ubuntu 18.04 or 20.04
  • c++ compiler (tested with g++-8.2)
  • cmake >= 3.8, to build the project
  • OpenCV >= 3.2, for image Input/Output routines

On a terminal, run this command to install needed packages:

apt-get install g++ cmake libopencv-dev

Creating a CMake Project

Create the cmake project to build the application as follows.

1 cmake_minimum_required(VERSION 3.5)
3 project(vpi_blur)
5 # This instructs cmake to look for the most recent
6 # vpi instance installed on the system.
7 find_package(vpi REQUIRED)
8 find_package(OpenCV REQUIRED)
10 # Creates the blur executable target
11 add_executable(vpi_blur main.cpp)
13 # It uses vpi and opencv. CMake will automatically
14 # set up the correct header and library directories,
15 # and make hello_work link to these libraries.
16 target_link_libraries(vpi_blur vpi opencv_core opencv_imgproc opencv_imgcodecs)

Note that cmake automatically sets up the VPI header and library paths for the executable. It is not necessary to manually define them.

Writing an Image Blurring Application

Copy the code below to a file named main.cpp. For simplicity, error checking is purposely left out. Consult the 2D convolution sample for a complete application with proper error handling. The comments explain how it works.

30 #include <opencv2/core/version.hpp>
31 #include <opencv2/imgproc/imgproc.hpp>
32 #if CV_MAJOR_VERSION >= 3
33 # include <opencv2/imgcodecs.hpp>
34 #else
35 # include <opencv2/highgui/highgui.hpp>
36 #endif
38 #include <iostream>
40 // All vpi headers are under directory vpi/
41 #include <vpi/OpenCVInterop.hpp>
43 #include <vpi/Image.h>
44 #include <vpi/Stream.h>
46 // Algorithms we'll need:
47 // - Image format conversion
48 // - Box filter for blurring
49 #include <vpi/algo/BoxFilter.h>
52 int main(int argc, char *argv[])
53 {
54  if (argc != 2)
55  {
56  std::cerr << "Must pass an input image to be blurred" << std::endl;
57  return 1;
58  }
60  // Phase 1: Initialization ---------------------------------
62  // First load the input image
63  cv::Mat cvImage = cv::imread(argv[1]);
64  if (cvImage.data == NULL)
65  {
66  std::cerr << "Can't open input image" << std::endl;
67  return 2;
68  }
70  // Now create the stream. Passing 0 allows algorithms submitted to it to run
71  // in any available backend.
72  VPIStream stream;
73  vpiStreamCreate(0, &stream);
75  // Now wrap the loaded image into a VPIImage object to be used by VPI.
76  // VPI will deduce the image type based on how many channels cvImage has.
77  // In this case, for OpenCV, which has 3 channels, it'll deduce to BGR8.
78  VPIImage image;
79  vpiImageCreateWrapperOpenCVMat(cvImage, 0, &image);
81  // Since we want to work with a grayscale version of input image, let's create
82  // an image that will store it.
83  VPIImage imageGray;
84  vpiImageCreate(cvImage.cols, cvImage.rows, VPI_IMAGE_FORMAT_U8, 0, &imageGray);
86  // Now create the output images, single unsigned 8-bit channel. The image lifetime is
87  // managed by VPI.
88  VPIImage blurred;
89  vpiImageCreate(cvImage.cols, cvImage.rows, VPI_IMAGE_FORMAT_U8, 0, &blurred);
91  // Phase 2: main processing --------------------------------------
93  // Pre-process the input, converting it from BGR8 to Grayscale
94  vpiSubmitConvertImageFormat(stream, VPI_BACKEND_CUDA, image, imageGray, NULL);
96  // Submit the algorithm task for processing along with inputs and outputs
97  // Parameters are:
98  // 1. the stream on which the algorithm will run
99  // 2. Which hardware backend will execute it. Here CUDA is specified, but it could be CPU or
100  // PVA (on Jetson Xavier devices). No further changes to the program are needed to have it executed
101  // on a different hardware.
102  // 3. input image to be blurred.
103  // 4. output image with the result.
104  // 5 and 6. box filter size, in this case, 5x5
105  // 7. Border extension for when the algorithm tries to sample pixels outside the image border.
106  // VPI_BORDER_ZERO will consider all such pixels as being 0.
107  vpiSubmitBoxFilter(stream, VPI_BACKEND_CUDA, imageGray, blurred, 5, 5, VPI_BORDER_ZERO);
109  // Block the current thread until until the stream finishes all tasks submitted to it up till now.
110  vpiStreamSync(stream);
112  // Finally retrieve the output image contents and output it to disk
114  // Lock output image to retrieve its data from cpu memory
115  VPIImageData outData;
118  // Construct an cv::Mat out of the retrieved data.
119  cv::Mat cvOut;
120  vpiImageDataExportOpenCVMat(outData, &cvOut);
122  // Now write it to disk
123  imwrite("tutorial_blurred.png", cvOut);
125  // Done handling output image, don't forget to unlock it.
126  vpiImageUnlock(blurred);
128  // Stage 3: clean up --------------------------------------
130  // Destroy all created objects.
131  vpiStreamDestroy(stream);
132  vpiImageDestroy(image);
133  vpiImageDestroy(imageGray);
134  vpiImageDestroy(blurred);
136  return 0;
137 }
Declares functions that implement the Box Filter algorithm.
Declares functions that handle image format conversion.
Single plane with one 8-bit unsigned integer channel.
Definition: ImageFormat.h:100
Functions and structures for dealing with VPI images.
Functions for handling OpenCV interoperability with VPI.
Declares functions dealing with VPI streams.
VPIStatus vpiSubmitBoxFilter(VPIStream stream, uint64_t backend, VPIImage input, VPIImage output, int32_t kernelWidth, int32_t kernelHeight, VPIBorderExtension border)
Runs a 2D box filter over an image.
VPIStatus vpiSubmitConvertImageFormat(VPIStream stream, uint64_t backend, VPIImage input, VPIImage output, const VPIConvertImageFormatParams *params)
Converts the image contents to the desired format, with optional scaling and offset.
void vpiImageDestroy(VPIImage img)
Destroy an image instance.
struct VPIImageImpl * VPIImage
A handle to an image.
Definition: Types.h:256
VPIStatus vpiImageLockData(VPIImage img, VPILockMode mode, VPIImageBufferType bufType, VPIImageData *data)
Acquires the lock on an image object and returns the image contents.
VPIStatus vpiImageCreate(int32_t width, int32_t height, VPIImageFormat fmt, uint64_t flags, VPIImage *img)
Create an empty image instance with the specified flags.
VPIStatus vpiImageUnlock(VPIImage img)
Releases the lock on an image object.
Host-accessible with planes in pitch-linear memory layout.
Definition: Image.h:172
Stores information about image characteristics and content.
Definition: Image.h:234
VPIStatus vpiImageCreateWrapperOpenCVMat(const cv::Mat &mat, VPIImageFormat fmt, uint64_t flags, VPIImage *img)
Wraps a cv::Mat in an VPIImage with the given image format.
VPIStatus vpiImageDataExportOpenCVMat(const VPIImageData &imgData, cv::Mat *mat)
Fills an existing cv::Mat with data from VPIImageData coming from a locked VPIImage.
struct VPIStreamImpl * VPIStream
A handle to a stream.
Definition: Types.h:250
VPIStatus vpiStreamSync(VPIStream stream)
Blocks the calling thread until all submitted commands in this stream queue are done (queue is empty)...
void vpiStreamDestroy(VPIStream stream)
Destroy a stream instance and deallocate all HW resources.
VPIStatus vpiStreamCreate(uint64_t flags, VPIStream *stream)
Create a stream instance.
CUDA backend.
Definition: Types.h:93
All pixels outside the image are considered to be zero.
Definition: Types.h:278
Lock memory only for reading.
Definition: Types.h:617

Building and Testing the Application

With everything set in place, execute:

cmake .

The executable vpi_blur is created in the same directory.

To run the application, execute:

./vpi_blur <image file name>

substituting <image file name> by some image on disk.