VPI - Vision Programming Interface

2.0 Release - Developer Preview

2D Image Convolution

Overview

The 2D Image Convolution application outputs an image with the edges of the input image, saving the result as an image file on disk. The user can define what backend will be used for processing.

Note
The output will be in grayscale as convolution is currently only supported for single-channel images.

Instructions

The command line parameters are:

<backend> <input image>

where

  • backend: either cpu, cuda or pva; it defines the backend that will perform the processing.
  • input image: input image file name; it accepts png, jpeg and possibly others.

Here's one example:

  • C++
    ./vpi_sample_01_convolve_2d cpu ../assets/kodim8.png
  • Python
    python3 main.py cpu ../assets/kodim8.png

This is using the CPU backend and one of the provided sample images. You can try with other images, respecting the constraints imposed by the algorithm.

Results

Input imageOutput image, edges

Source Code

For convenience, here's the code that is also installed in the samples directory.

Language:
27 import sys
28 import vpi
29 import numpy as np
30 from PIL import Image
31 from argparse import ArgumentParser
32 
33 # Parse command line arguments
34 parser = ArgumentParser()
35 parser.add_argument('backend', choices=['cpu','cuda','pva'],
36  help='Backend to be used for processing')
37 
38 parser.add_argument('input',
39  help='Image to be used as input')
40 
41 args = parser.parse_args();
42 
43 if args.backend == 'cpu':
44  backend = vpi.Backend.CPU
45 elif args.backend == 'cuda':
46  backend = vpi.Backend.CUDA
47 else:
48  assert args.backend == 'pva'
49  backend = vpi.Backend.PVA
50 
51 # Load input into a vpi.Image
52 input = vpi.asimage(np.asarray(Image.open(args.input)))
53 
54 # Convert it to grayscale
55 input = input.convert(vpi.Format.U8, backend=vpi.Backend.CUDA)
56 
57 # Define a simple edge detection kernel
58 kernel = [[ 1, 0, -1],
59  [ 0, 0, 0],
60  [-1, 0, 1]]
61 
62 # Using the chosen backend,
63 with backend:
64  # Run input through the convolution filter
65  output = input.convolution(kernel, border=vpi.Border.ZERO)
66 
67 # Save result to disk
68 Image.fromarray(output.cpu()).save('edges_python'+str(sys.version_info[0])+'_'+args.backend+'.png')
29 #include <opencv2/core/version.hpp>
30 #include <opencv2/imgproc/imgproc.hpp>
31 #if CV_MAJOR_VERSION >= 3
32 # include <opencv2/imgcodecs.hpp>
33 #else
34 # include <opencv2/highgui/highgui.hpp>
35 #endif
36 
37 #include <vpi/OpenCVInterop.hpp>
38 
39 #include <vpi/Image.h>
40 #include <vpi/Status.h>
41 #include <vpi/Stream.h>
43 #include <vpi/algo/Convolution.h>
44 
45 #include <cstring> // for memset
46 #include <iostream>
47 #include <sstream>
48 
49 #define CHECK_STATUS(STMT) \
50  do \
51  { \
52  VPIStatus status = (STMT); \
53  if (status != VPI_SUCCESS) \
54  { \
55  char buffer[VPI_MAX_STATUS_MESSAGE_LENGTH]; \
56  vpiGetLastStatusMessage(buffer, sizeof(buffer)); \
57  std::ostringstream ss; \
58  ss << vpiStatusGetName(status) << ": " << buffer; \
59  throw std::runtime_error(ss.str()); \
60  } \
61  } while (0);
62 
63 int main(int argc, char *argv[])
64 {
65  // OpenCV image that will be wrapped by a VPIImage.
66  // Define it here so that it's destroyed *after* wrapper is destroyed
67  cv::Mat cvImage;
68 
69  // VPI objects that will be used
70  VPIImage image = NULL;
71  VPIImage imageBGR = NULL;
72  VPIImage gradient = NULL;
73  VPIStream stream = NULL;
74 
75  int retval = 0;
76 
77  try
78  {
79  if (argc != 3)
80  {
81  throw std::runtime_error(std::string("Usage: ") + argv[0] + " <cpu|pva|cuda> <input image>");
82  }
83 
84  std::string strBackend = argv[1];
85  std::string strInputFileName = argv[2];
86 
87  // Load the input image
88  cvImage = cv::imread(strInputFileName);
89  if (cvImage.empty())
90  {
91  throw std::runtime_error("Can't open '" + strInputFileName + "'");
92  }
93 
94  // We can't use cv::IMREAD_GRAYSCALE when opening the input file because the
95  // color to grayscale conversion used differs between OpenCV-2.4 and OpenCV>=3.0,
96  // yielding different image content.
97 
98  // Now parse the backend
99  VPIBackend backend;
100 
101  if (strBackend == "cpu")
102  {
103  backend = VPI_BACKEND_CPU;
104  }
105  else if (strBackend == "cuda")
106  {
107  backend = VPI_BACKEND_CUDA;
108  }
109  else if (strBackend == "pva")
110  {
111  backend = VPI_BACKEND_PVA;
112  }
113  else
114  {
115  throw std::runtime_error("Backend '" + strBackend +
116  "' not recognized, it must be either cpu, cuda or pva.");
117  }
118 
119  // Create the stream for any backend.
120  CHECK_STATUS(vpiStreamCreate(0, &stream));
121 
122  // We now wrap the loaded image into a VPIImage object to be used by VPI.
123  CHECK_STATUS(vpiImageCreateWrapperOpenCVMat(cvImage, 0, &imageBGR));
124 
125  // Now create the input image as a single unsigned 8-bit channel
126  CHECK_STATUS(vpiImageCreate(cvImage.cols, cvImage.rows, VPI_IMAGE_FORMAT_U8, 0, &image));
127 
128  // Convert the loaded image to grayscale
129  CHECK_STATUS(vpiSubmitConvertImageFormat(stream, VPI_BACKEND_CUDA, imageBGR, image, NULL));
130 
131  // Now create the output image, single unsigned 8-bit channel.
132  CHECK_STATUS(vpiImageCreate(cvImage.cols, cvImage.rows, VPI_IMAGE_FORMAT_U8, 0, &gradient));
133 
134  // Define the convolution filter, a simple edge detector.
135  float kernel[3 * 3] = {1, 0, -1, 0, 0, 0, -1, 0, 1};
136 
137  // Submit it for processing passing the input image the result image that will store the gradient.
138  CHECK_STATUS(vpiSubmitConvolution(stream, backend, image, gradient, kernel, 3, 3, VPI_BORDER_ZERO));
139 
140  // Wait until the algorithm finishes processing
141  CHECK_STATUS(vpiStreamSync(stream));
142 
143  // Now let's retrieve the output image contents and output it to disk
144  {
145  // Lock output image to retrieve its data.
146  VPIImageData outData;
147  CHECK_STATUS(vpiImageLockData(gradient, VPI_LOCK_READ, VPI_IMAGE_BUFFER_HOST_PITCH_LINEAR, &outData));
148 
149  // Returned data consists of host-accessible memory buffers in pitch-linear layout.
151 
152  const VPIImageBufferPitchLinear &outPitch = outData.buffer.pitch;
153 
154  cv::Mat cvOut(outPitch.planes[0].height, outPitch.planes[0].width, CV_8UC1, outPitch.planes[0].data,
155  outPitch.planes[0].pitchBytes);
156  imwrite("edges_" + strBackend + ".png", cvOut);
157 
158  // Done handling output image, don't forget to unlock it.
159  CHECK_STATUS(vpiImageUnlock(gradient));
160  }
161  }
162  catch (std::exception &e)
163  {
164  std::cerr << e.what() << std::endl;
165  retval = 1;
166  }
167 
168  // Clean up
169 
170  // Make sure stream is synchronized before destroying the objects
171  // that might still be in use.
172  if (stream != NULL)
173  {
174  vpiStreamSync(stream);
175  }
176 
177  vpiImageDestroy(image);
178  vpiImageDestroy(imageBGR);
179  vpiImageDestroy(gradient);
180 
181  vpiStreamDestroy(stream);
182 
183  return retval;
184 }
Declares functions that handle image format conversion.
Declares functions to perform image filtering with convolution kernels.
#define VPI_IMAGE_FORMAT_U8
Single plane with one 8-bit unsigned integer channel.
Definition: ImageFormat.h:100
Functions and structures for dealing with VPI images.
Functions for handling OpenCV interoperability with VPI.
Declaration of VPI status codes handling functions.
Declares functions dealing with VPI streams.
VPIStatus vpiSubmitConvertImageFormat(VPIStream stream, uint64_t backend, VPIImage input, VPIImage output, const VPIConvertImageFormatParams *params)
Converts the image contents to the desired format, with optional scaling and offset.
VPIStatus vpiSubmitConvolution(VPIStream stream, uint64_t backend, VPIImage input, VPIImage output, const float *kernelData, int32_t kernelWidth, int32_t kernelHeight, VPIBorderExtension border)
Runs a generic 2D convolution over an image.
VPIImageBuffer buffer
Stores the image contents.
Definition: Image.h:237
VPIImagePlanePitchLinear planes[VPI_MAX_PLANE_COUNT]
Data of all image planes in pitch-linear layout.
Definition: Image.h:160
VPIImageBufferPitchLinear pitch
Image stored in pitch-linear layout.
Definition: Image.h:206
void * data
Pointer to the first row of this plane.
Definition: Image.h:141
VPIImageBufferType bufferType
Type of image buffer.
Definition: Image.h:234
int32_t height
Height of this plane in pixels.
Definition: Image.h:123
int32_t width
Width of this plane in pixels.
Definition: Image.h:119
int32_t pitchBytes
Difference in bytes of beginning of one row and the beginning of the previous.
Definition: Image.h:134
void vpiImageDestroy(VPIImage img)
Destroy an image instance.
struct VPIImageImpl * VPIImage
A handle to an image.
Definition: Types.h:256
VPIStatus vpiImageLockData(VPIImage img, VPILockMode mode, VPIImageBufferType bufType, VPIImageData *data)
Acquires the lock on an image object and returns the image contents.
VPIStatus vpiImageCreate(int32_t width, int32_t height, VPIImageFormat fmt, uint64_t flags, VPIImage *img)
Create an empty image instance with the specified flags.
VPIStatus vpiImageUnlock(VPIImage img)
Releases the lock on an image object.
@ VPI_IMAGE_BUFFER_HOST_PITCH_LINEAR
Host-accessible with planes in pitch-linear memory layout.
Definition: Image.h:172
Stores the image plane contents.
Definition: Image.h:150
Stores information about image characteristics and content.
Definition: Image.h:230
VPIStatus vpiImageCreateWrapperOpenCVMat(const cv::Mat &mat, VPIImageFormat fmt, uint64_t flags, VPIImage *img)
Wraps a cv::Mat in an VPIImage with the given image format.
struct VPIStreamImpl * VPIStream
A handle to a stream.
Definition: Types.h:250
VPIStatus vpiStreamSync(VPIStream stream)
Blocks the calling thread until all submitted commands in this stream queue are done (queue is empty)...
VPIBackend
VPI Backend types.
Definition: Types.h:91
void vpiStreamDestroy(VPIStream stream)
Destroy a stream instance and deallocate all HW resources.
VPIStatus vpiStreamCreate(uint64_t flags, VPIStream *stream)
Create a stream instance.
@ VPI_BACKEND_CUDA
CUDA backend.
Definition: Types.h:93
@ VPI_BACKEND_PVA
PVA backend.
Definition: Types.h:94
@ VPI_BACKEND_CPU
CPU backend.
Definition: Types.h:92
@ VPI_BORDER_ZERO
All pixels outside the image are considered to be zero.
Definition: Types.h:278
@ VPI_LOCK_READ
Lock memory only for reading.
Definition: Types.h:435