VPI - Vision Programming Interface

1.0 Release

Quick Tutorial

This tutorial provides step-by-step procedures for building an application that demonstrates VPI programming concepts. With this tutorial a simple grayscale image blurring command line application that runs on CUDA is created.

Requirements

Creating the application requires:

  • c++ compiler (g++ or clang++ works, testing was performed with version g++-7 and clang++-7)
  • cmake >= 3.8, to build the project
  • OpenCV >= 3.0, for image Input/Output

On Ubuntu 18.04, install these packages:

apt-get install g++ cmake libopencv-dev

Creating a CMake Project

Create the cmake project to build the application as follows.

1 cmake_minimum_required(VERSION 3.5)
2 
3 project(vpi_blur)
4 
5 # This instructs cmake to look for the most recent
6 # vpi instance installed on the system.
7 find_package(vpi REQUIRED)
8 find_package(OpenCV REQUIRED)
9 
10 # Creates the blur executable target
11 add_executable(vpi_blur main.cpp)
12 
13 # It uses vpi and opencv. CMake will automatically
14 # set up the correct header and library directories,
15 # and make hello_work link to these libraries.
16 target_link_libraries(vpi_blur vpi opencv_core)
17 
18 # OpenCV < 3 uses a different library for image i/o
19 if(OpenCV_VERSION VERSION_LESS 3)
20  target_link_libraries(vpi_blur opencv_highgui)
21 else()
22  target_link_libraries(vpi_blur opencv_imgcodecs)
23 endif()

Note that cmake automatically sets up the VPI header and library paths for the executable. It is not necessary to manually define them.

Writing an Image Blurring Application

Write the C++ code that gets one image file from command line, blurs it, and writes the results back to disk.

Copy the provided code to a file named main.cpp. For simplicity, error checking is purposely left out. Consult the 2D convolution sample for a complete application with proper error handling.

1 /*
2 * Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
3 *
4 * Redistribution and use in source and binary forms, with or without
5 * modification, are permitted provided that the following conditions
6 * are met:
7 * * Redistributions of source code must retain the above copyright
8 * notice, this list of conditions and the following disclaimer.
9 * * Redistributions in binary form must reproduce the above copyright
10 * notice, this list of conditions and the following disclaimer in the
11 * documentation and/or other materials provided with the distribution.
12 * * Neither the name of NVIDIA CORPORATION nor the names of its
13 * contributors may be used to endorse or promote products derived
14 * from this software without specific prior written permission.
15 *
16 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
17 * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
19 * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
20 * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
21 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
22 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
23 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
24 * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
25 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
26 * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
27 */
28 
29 // For image I/O
30 #include <opencv2/core/version.hpp>
31 #if CV_MAJOR_VERSION >= 3
32 # include <opencv2/imgcodecs.hpp>
33 #else
34 # include <opencv2/highgui/highgui.hpp>
35 #endif
36 
37 #include <iostream>
38 
39 // All vpi headers are under directory vpi/
40 #include <vpi/OpenCVInterop.hpp>
41 
42 #include <vpi/Image.h>
43 #include <vpi/Stream.h>
44 
45 // Algorithm we'll need. Blurring will be performed by a box filter.
46 #include <vpi/algo/BoxFilter.h>
47 
48 int main(int argc, char *argv[])
49 {
50  if (argc != 2)
51  {
52  std::cerr << "Must pass an input image to be blurred" << std::endl;
53  return 1;
54  }
55 
56  // Phase 1: Initialization ---------------------------------
57 
58  // First load the input image
59  cv::Mat cvImage = cv::imread(argv[1], cv::IMREAD_GRAYSCALE);
60  if (cvImage.data == NULL)
61  {
62  std::cerr << "Can't open input image" << std::endl;
63  return 2;
64  }
65 
66  // Now create the stream. Passing 0 allows algorithms submitted to it to run
67  // in any available backend.
68  VPIStream stream;
69  vpiStreamCreate(0, &stream);
70 
71  // Now wrap the loaded image into a VPIImage object to be used by VPI.
72  VPIImage image;
73  vpiImageCreateOpenCVMatWrapper(cvImage, 0, &image);
74 
75  // Now create the output images, single unsigned 8-bit channel. The image lifetime is
76  // managed by VPI.
77  VPIImage blurred;
78  vpiImageCreate(cvImage.cols, cvImage.rows, VPI_IMAGE_FORMAT_U8, 0, &blurred);
79 
80  // Phase 2: main processing --------------------------------------
81 
82  // Submit the algorithm task for processing along with inputs and outputs
83  // Parameters are:
84  // 1. the stream on which the algorithm will run
85  // 2. Which hardware backend will execute it. Here CUDA is specified, but it could be CPU or
86  // PVA (on Jetson Xavier devices). No further changes to the program are needed to have it executed
87  // on a different hardware.
88  // 3. input image to be blurred.
89  // 4. output image with the result.
90  // 5 and 6. box filter size, in this case, 5x5
91  // 7. Border extension for when the algorithm tries to sample pixels outside the image border.
92  // VPI_BORDER_ZERO will consider all such pixels as being 0.
93  vpiSubmitBoxFilter(stream, VPI_BACKEND_CUDA, image, blurred, 5, 5, VPI_BORDER_ZERO);
94 
95  // Block the current thread until until the stream finishes all tasks submitted to it up till now.
96  vpiStreamSync(stream);
97 
98  // Finally retrieve the output image contents and output it to disk
99 
100  // Lock output image to retrieve its data from cpu memory
101  VPIImageData outData;
102  vpiImageLock(blurred, VPI_LOCK_READ, &outData);
103 
104  // Construct an cv::Mat out of the retrieved data.
105  cv::Mat cvOut(outData.planes[0].height, outData.planes[0].width, CV_8UC1, outData.planes[0].data,
106  outData.planes[0].pitchBytes);
107  // Just write it to disk
108  imwrite("tutorial_blurred.png", cvOut);
109 
110  // Done handling output image, don't forget to unlock it.
111  vpiImageUnlock(blurred);
112 
113  // Stage 3: clean up --------------------------------------
114 
115  // Destroy all created objects.
116  vpiStreamDestroy(stream);
117  vpiImageDestroy(image);
118  vpiImageDestroy(blurred);
119 
120  return 0;
121 }
Declares functions that implement the Box Filter algorithm.
Functions and structures for dealing with VPI images.
Functions for handling OpenCV interoperability with VPI.
Declares functions dealing with VPI streams.
VPIStatus vpiSubmitBoxFilter(VPIStream stream, uint32_t backend, VPIImage input, VPIImage output, int32_t kernelSizeX, int32_t kernelSizeY, VPIBorderExtension border)
Runs a 2D box filter over an image.
@ VPI_IMAGE_FORMAT_U8
Single plane with one 8-bit unsigned integer channel.
Definition: ImageFormat.h:104
int32_t height
Height of this plane in pixels.
Definition: Image.h:138
int32_t width
Width of this plane in pixels.
Definition: Image.h:137
void * data
Pointer to the first row of this plane.
Definition: Image.h:147
int32_t pitchBytes
Difference in bytes of beginning of one row and the beginning of the previous.
Definition: Image.h:139
VPIImagePlane planes[VPI_MAX_PLANE_COUNT]
Data of all image planes.
Definition: Image.h:166
VPIStatus vpiImageLock(VPIImage img, VPILockMode mode, VPIImageData *hostData)
Acquires the lock on an image object and returns a pointer to the image planes.
void vpiImageDestroy(VPIImage img)
Destroy an image instance.
struct VPIImageImpl * VPIImage
A handle to an image.
Definition: Types.h:197
VPIStatus vpiImageCreate(int32_t width, int32_t height, VPIImageFormat fmt, uint32_t flags, VPIImage *img)
Create an empty image instance with the specified flags.
VPIStatus vpiImageUnlock(VPIImage img)
Releases the lock on an image object.
Stores information about image characteristics and content.
Definition: Image.h:159
VPIStatus vpiImageCreateOpenCVMatWrapper(const cv::Mat &mat, VPIImageFormat fmt, uint32_t flags, VPIImage *img)
Wraps a cv::Mat in an VPIImage with the given image format.
struct VPIStreamImpl * VPIStream
A handle to a stream.
Definition: Types.h:191
VPIStatus vpiStreamSync(VPIStream stream)
Blocks the calling thread until all submitted commands in this stream queue are done (queue is empty)...
void vpiStreamDestroy(VPIStream stream)
Destroy a stream instance and deallocate all HW resources.
VPIStatus vpiStreamCreate(uint32_t flags, VPIStream *stream)
Create a stream instance.
@ VPI_BACKEND_CUDA
CUDA backend.
Definition: Types.h:92
@ VPI_BORDER_ZERO
All pixels outside the image are considered to be zero.
Definition: Types.h:219
@ VPI_LOCK_READ
Lock memory only for reading.
Definition: Types.h:348

Building and Testing the Application

With everything set in place, execute:

cmake .
make

The executable vpi_blur is created in the same directory.

To run the application, execute:

./vpi_blur <image file name>

substituting <image file name> by some image on disk.

A sample input image and the resulting blurred image is displayed. Note that although the input in in color, OpenCV converts the to grayscale prior passing it to VPI which results in grayscale output.

Input image Output image, blurred