VPI - Vision Programming Interface

1.0 Release

Image Resampling

Overview

The Rescale application rescales the input image by first applying a low-pass filter to avoid aliasing, then doing a downsampling. The resulting image has half of input's width and one third of input's height. The result is then saved to disk.

This sample shows the following:

  • Creating and destroying a VPI stream.
  • Wrapping a OpenCV cv::Mat image to be used with VPI.
  • Creating a VPI-managed 2D image.
  • Create a pipeline with two operations, GaussianFilter and Rescale.
  • Simple stream synchronization.
  • Image locking to access its contents from CPU side.
  • Error handling.
  • Environment clean up.

Instructions

The usage is:

./vpi_sample_04_rescale <backend> <input image>

where

  • backend: either cpu, cuda or pva; it defines the backend that will perform the processing.
  • input image: input image file name to be downsampled, it accepts png, jpeg and possibly others.

Here's one example:

./vpi_sample_04_rescale cuda ../assets/kodim8.png

This is using the CUDA backend and one of the provided sample images. You can try with other images, respecting the constraints imposed by the algorithm.

Results

Input imageOutput image, downsampled

Source Code

For convenience, here's the code that is also installed in the samples directory.

1 /*
2 * Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
3 *
4 * Redistribution and use in source and binary forms, with or without
5 * modification, are permitted provided that the following conditions
6 * are met:
7 * * Redistributions of source code must retain the above copyright
8 * notice, this list of conditions and the following disclaimer.
9 * * Redistributions in binary form must reproduce the above copyright
10 * notice, this list of conditions and the following disclaimer in the
11 * documentation and/or other materials provided with the distribution.
12 * * Neither the name of NVIDIA CORPORATION nor the names of its
13 * contributors may be used to endorse or promote products derived
14 * from this software without specific prior written permission.
15 *
16 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
17 * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
19 * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
20 * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
21 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
22 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
23 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
24 * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
25 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
26 * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
27 */
28 
29 #include <opencv2/core/version.hpp>
30 #if CV_MAJOR_VERSION >= 3
31 # include <opencv2/imgcodecs.hpp>
32 #else
33 # include <opencv2/highgui/highgui.hpp>
34 #endif
35 
36 #include <vpi/OpenCVInterop.hpp>
37 
38 #include <vpi/Image.h>
39 #include <vpi/Status.h>
40 #include <vpi/Stream.h>
42 #include <vpi/algo/Rescale.h>
43 
44 #include <cstring> // for memset
45 #include <iostream>
46 #include <sstream>
47 
48 #define CHECK_STATUS(STMT) \
49  do \
50  { \
51  VPIStatus status = (STMT); \
52  if (status != VPI_SUCCESS) \
53  { \
54  char buffer[VPI_MAX_STATUS_MESSAGE_LENGTH]; \
55  vpiGetLastStatusMessage(buffer, sizeof(buffer)); \
56  std::ostringstream ss; \
57  ss << vpiStatusGetName(status) << ": " << buffer; \
58  throw std::runtime_error(ss.str()); \
59  } \
60  } while (0);
61 
62 int main(int argc, char *argv[])
63 {
64  VPIImage image = NULL;
65  VPIImage imageNV12 = NULL;
66  VPIImage outputNV12 = NULL;
67  VPIImage output = NULL;
68  VPIStream stream = NULL;
69 
70  int retval = 0;
71 
72  try
73  {
74  if (argc != 3)
75  {
76  throw std::runtime_error(std::string("Usage: ") + argv[0] + " <cpu|vic|cuda> <input image>");
77  }
78 
79  std::string strBackend = argv[1];
80  std::string strInputFileName = argv[2];
81 
82  // Load the input image
83  cv::Mat cvImage = cv::imread(strInputFileName);
84  if (cvImage.empty())
85  {
86  throw std::runtime_error("Can't open '" + strInputFileName + "'");
87  }
88 
89  assert(cvImage.type() == CV_8UC3);
90 
91  // Now parse the backend
92  VPIBackend backendType;
93 
94  if (strBackend == "cpu")
95  {
96  backendType = VPI_BACKEND_CPU;
97  }
98  else if (strBackend == "cuda")
99  {
100  backendType = VPI_BACKEND_CUDA;
101  }
102  else if (strBackend == "vic")
103  {
104  backendType = VPI_BACKEND_VIC;
105  }
106  else
107  {
108  throw std::runtime_error("Backend '" + strBackend + "' not recognized, it must be either cpu, cuda or vic");
109  }
110 
111  // 1. Initialization phase ---------------------------------------
112 
113  // Create the stream for the given backend. We'll also enable CUDA for gaussian filter.
114  CHECK_STATUS(vpiStreamCreate(backendType | VPI_BACKEND_CUDA, &stream));
115 
116  // We now wrap the loaded image into a VPIImage object to be used by VPI.
117  // VPI won't make a copy of it, so the original
118  // image must be in scope at all times.
119  CHECK_STATUS(vpiImageCreateOpenCVMatWrapper(cvImage, 0, &image));
120 
121  // Create a temporary image to hold the input converted to NV12.
122  CHECK_STATUS(vpiImageCreate(cvImage.cols, cvImage.rows, VPI_IMAGE_FORMAT_NV12_ER, 0, &imageNV12));
123 
124  // Now create the output image.
125  CHECK_STATUS(vpiImageCreate(cvImage.cols / 2, cvImage.rows / 3, VPI_IMAGE_FORMAT_NV12_ER, 0, &outputNV12));
126 
127  // And the output image converted back to BGR8
128  CHECK_STATUS(vpiImageCreate(cvImage.cols / 2, cvImage.rows / 3, VPI_IMAGE_FORMAT_BGR8, 0, &output));
129 
130  // 2. Computation phase ---------------------------------------
131 
132  // Convert input from BGR8 to NV12
133  CHECK_STATUS(vpiSubmitConvertImageFormat(stream, VPI_BACKEND_CUDA, image, imageNV12, NULL));
134 
135  // Now we downsample
136  CHECK_STATUS(
137  vpiSubmitRescale(stream, backendType, imageNV12, outputNV12, VPI_INTERP_LINEAR, VPI_BORDER_CLAMP, 0));
138 
139  // Finally, convert the result back to BGR8
140  CHECK_STATUS(vpiSubmitConvertImageFormat(stream, VPI_BACKEND_CUDA, outputNV12, output, NULL));
141 
142  // Wait until the algorithm finishes processing
143  CHECK_STATUS(vpiStreamSync(stream));
144 
145  // Now let's retrieve the output image contents and output it to disk
146  {
147  VPIImageData data;
148  CHECK_STATUS(vpiImageLock(output, VPI_LOCK_READ, &data));
149 
150  // Lock output image to retrieve its data on cpu memory
151  VPIImageData outData;
152  CHECK_STATUS(vpiImageLock(output, VPI_LOCK_READ, &outData));
153 
154  cv::Mat cvOut(outData.planes[0].height, outData.planes[0].width, CV_8UC3, outData.planes[0].data,
155  outData.planes[0].pitchBytes);
156  imwrite("scaled_" + strBackend + ".png", cvOut);
157 
158  // Done handling output image, don't forget to unlock it.
159  CHECK_STATUS(vpiImageUnlock(output));
160  }
161  }
162  catch (std::exception &e)
163  {
164  std::cerr << e.what() << std::endl;
165  retval = 1;
166  }
167 
168  // Clean up
169 
170  // Make sure stream is synchronized before destroying the objects
171  // that might still be in use.
172  if (stream != NULL)
173  {
174  vpiStreamSync(stream);
175  }
176 
177  vpiImageDestroy(image);
178  vpiImageDestroy(imageNV12);
179  vpiImageDestroy(output);
180  vpiStreamDestroy(stream);
181 
182  return retval;
183 }
Declares functions that handle image format conversion.
Functions and structures for dealing with VPI images.
Functions for handling OpenCV interoperability with VPI.
Declares functions that implement the Rescale algorithm.
Declaration of VPI status codes handling functions.
Declares functions dealing with VPI streams.
VPIStatus vpiSubmitConvertImageFormat(VPIStream stream, uint32_t backend, VPIImage input, VPIImage output, const VPIConvertImageFormatParams *params)
Converts the image contents to the desired format, with optional scaling and offset.
@ VPI_IMAGE_FORMAT_BGR8
Single plane with interleaved BGR 8-bit channel.
Definition: ImageFormat.h:200
@ VPI_IMAGE_FORMAT_NV12_ER
YUV420sp 8-bit pitch-linear format with full range.
Definition: ImageFormat.h:152
int32_t height
Height of this plane in pixels.
Definition: Image.h:138
int32_t width
Width of this plane in pixels.
Definition: Image.h:137
void * data
Pointer to the first row of this plane.
Definition: Image.h:147
int32_t pitchBytes
Difference in bytes of beginning of one row and the beginning of the previous.
Definition: Image.h:139
VPIImagePlane planes[VPI_MAX_PLANE_COUNT]
Data of all image planes.
Definition: Image.h:166
VPIStatus vpiImageLock(VPIImage img, VPILockMode mode, VPIImageData *hostData)
Acquires the lock on an image object and returns a pointer to the image planes.
void vpiImageDestroy(VPIImage img)
Destroy an image instance.
struct VPIImageImpl * VPIImage
A handle to an image.
Definition: Types.h:197
VPIStatus vpiImageCreate(int32_t width, int32_t height, VPIImageFormat fmt, uint32_t flags, VPIImage *img)
Create an empty image instance with the specified flags.
VPIStatus vpiImageUnlock(VPIImage img)
Releases the lock on an image object.
Stores information about image characteristics and content.
Definition: Image.h:159
VPIStatus vpiImageCreateOpenCVMatWrapper(const cv::Mat &mat, VPIImageFormat fmt, uint32_t flags, VPIImage *img)
Wraps a cv::Mat in an VPIImage with the given image format.
VPIStatus vpiSubmitRescale(VPIStream stream, uint32_t backend, VPIImage input, VPIImage output, VPIInterpolationType interpolationType, VPIBorderExtension border, uint32_t flags)
Changes the size and scale of a 2D image.
struct VPIStreamImpl * VPIStream
A handle to a stream.
Definition: Types.h:191
VPIStatus vpiStreamSync(VPIStream stream)
Blocks the calling thread until all submitted commands in this stream queue are done (queue is empty)...
VPIBackend
VPI Backend types.
Definition: Types.h:90
void vpiStreamDestroy(VPIStream stream)
Destroy a stream instance and deallocate all HW resources.
VPIStatus vpiStreamCreate(uint32_t flags, VPIStream *stream)
Create a stream instance.
@ VPI_BACKEND_CUDA
CUDA backend.
Definition: Types.h:92
@ VPI_BACKEND_VIC
VIC backend.
Definition: Types.h:94
@ VPI_BACKEND_CPU
CPU backend.
Definition: Types.h:91
@ VPI_BORDER_CLAMP
Border pixels are repeated indefinitely.
Definition: Types.h:220
@ VPI_INTERP_LINEAR
Linear interpolation.
Definition: Interpolation.h:93
@ VPI_LOCK_READ
Lock memory only for reading.
Definition: Types.h:348