VPI - Vision Programming Interface

3.2 Release

Image Resampling

Overview

The Rescale application rescales the input image by first applying a low-pass filter to avoid aliasing, then doing a downsampling. The resulting image has half of input's width and one third of input's height. The result is then saved to disk.

Instructions

The command line parameters are:

<backend> <input image>

where

  • backend: either cpu, cuda or vic; it defines the backend that will perform the processing.
  • input image: input image file name to be downsampled, it accepts png, jpeg and possibly others.

Here's one example:

  • C++
    ./vpi_sample_04_rescale cuda ../assets/kodim08.png
  • Python
    python3 main.py cuda ../assets/kodim08.png

This is using the CUDA backend and one of the provided sample images. You can try with other images, respecting the constraints imposed by the algorithm.

Results

Input imageOutput image, downsampled

Source Code

For convenience, here's the code that is also installed in the samples directory.

Language:
27 import sys
28 import vpi
29 import numpy as np
30 from PIL import Image
31 from argparse import ArgumentParser
32 
33 # Parse command line arguments
34 parser = ArgumentParser()
35 parser.add_argument('backend', choices=['cpu','cuda','vic'],
36  help='Backend to be used for processing')
37 
38 parser.add_argument('input',
39  help='Image to be used as input')
40 
41 args = parser.parse_args();
42 
43 if args.backend == 'cpu':
44  backend = vpi.Backend.CPU
45 elif args.backend == 'cuda':
46  backend = vpi.Backend.CUDA
47 else:
48  assert args.backend == 'vic'
49  backend = vpi.Backend.VIC
50 
51 # Load input into a vpi.Image
52 try:
53  input = vpi.asimage(np.asarray(Image.open(args.input)))
54 except IOError:
55  sys.exit("Input file not found")
56 except:
57  sys.exit("Error with input file")
58 
59 # Using the chosen backend,
60 with backend:
61  # First convert input to NV12_ER.
62  # We're overriding the default backend with CUDA.
63  temp = input.convert(vpi.Format.NV12_ER, backend=vpi.Backend.CUDA)
64 
65  # Rescale the image using the chosen backend
66  temp = temp.rescale((input.width//2, input.height//3))
67 
68  # Convert result back to input's format
69  output = temp.convert(input.format, backend=vpi.Backend.CUDA)
70 
71 # Save result to disk
72 Image.fromarray(output.cpu()).save('scaled_python'+str(sys.version_info[0])+'_'+args.backend+'.png')
29 #include <opencv2/core/version.hpp>
30 #if CV_MAJOR_VERSION >= 3
31 # include <opencv2/imgcodecs.hpp>
32 #else
33 # include <opencv2/highgui/highgui.hpp>
34 #endif
35 
36 #include <vpi/OpenCVInterop.hpp>
37 
38 #include <vpi/Image.h>
39 #include <vpi/Status.h>
40 #include <vpi/Stream.h>
42 #include <vpi/algo/Rescale.h>
43 
44 #include <cassert>
45 #include <cstring> // for memset
46 #include <iostream>
47 #include <sstream>
48 
49 #define CHECK_STATUS(STMT) \
50  do \
51  { \
52  VPIStatus status = (STMT); \
53  if (status != VPI_SUCCESS) \
54  { \
55  char buffer[VPI_MAX_STATUS_MESSAGE_LENGTH]; \
56  vpiGetLastStatusMessage(buffer, sizeof(buffer)); \
57  std::ostringstream ss; \
58  ss << vpiStatusGetName(status) << ": " << buffer; \
59  throw std::runtime_error(ss.str()); \
60  } \
61  } while (0);
62 
63 int main(int argc, char *argv[])
64 {
65  // OpenCV image that will be wrapped by a VPIImage.
66  // Define it here so that it's destroyed *after* wrapper is destroyed
67  cv::Mat cvImage;
68 
69  // VPI objects that will be used
70  VPIImage image = NULL;
71  VPIImage imageNV12 = NULL;
72  VPIImage outputNV12 = NULL;
73  VPIImage output = NULL;
74  VPIStream stream = NULL;
75 
76  int retval = 0;
77 
78  try
79  {
80  if (argc != 3)
81  {
82  throw std::runtime_error(std::string("Usage: ") + argv[0] + " <cpu|vic|cuda> <input image>");
83  }
84 
85  std::string strBackend = argv[1];
86  std::string strInputFileName = argv[2];
87 
88  // Load the input image
89  cvImage = cv::imread(strInputFileName);
90  if (cvImage.empty())
91  {
92  throw std::runtime_error("Can't open '" + strInputFileName + "'");
93  }
94 
95  assert(cvImage.type() == CV_8UC3);
96 
97  // Now parse the backend
98  VPIBackend backend;
99 
100  if (strBackend == "cpu")
101  {
102  backend = VPI_BACKEND_CPU;
103  }
104  else if (strBackend == "cuda")
105  {
106  backend = VPI_BACKEND_CUDA;
107  }
108  else if (strBackend == "vic")
109  {
110  backend = VPI_BACKEND_VIC;
111  }
112  else
113  {
114  throw std::runtime_error("Backend '" + strBackend + "' not recognized, it must be either cpu, cuda or vic");
115  }
116 
117  // 1. Initialization phase ---------------------------------------
118 
119  // Create the stream for the given backend. We'll also enable CUDA for gaussian filter.
120  CHECK_STATUS(vpiStreamCreate(backend | VPI_BACKEND_CUDA, &stream));
121 
122  // We now wrap the loaded image into a VPIImage object to be used by VPI.
123  // VPI won't make a copy of it, so the original
124  // image must be in scope at all times.
125  CHECK_STATUS(vpiImageCreateWrapperOpenCVMat(cvImage, 0, &image));
126 
127  // Create a temporary image to hold the input converted to NV12.
128  CHECK_STATUS(vpiImageCreate(cvImage.cols, cvImage.rows, VPI_IMAGE_FORMAT_NV12_ER, 0, &imageNV12));
129 
130  // Now create the output image.
131  CHECK_STATUS(vpiImageCreate(cvImage.cols / 2, cvImage.rows / 3, VPI_IMAGE_FORMAT_NV12_ER, 0, &outputNV12));
132 
133  // And the output image converted back to BGR8
134  CHECK_STATUS(vpiImageCreate(cvImage.cols / 2, cvImage.rows / 3, VPI_IMAGE_FORMAT_BGR8, 0, &output));
135 
136  // 2. Computation phase ---------------------------------------
137 
138  // Convert input from BGR8 to NV12
139  CHECK_STATUS(vpiSubmitConvertImageFormat(stream, VPI_BACKEND_CUDA, image, imageNV12, NULL));
140 
141  // Now we downsample
142  CHECK_STATUS(vpiSubmitRescale(stream, backend, imageNV12, outputNV12, VPI_INTERP_LINEAR, VPI_BORDER_CLAMP, 0));
143 
144  // Finally, convert the result back to BGR8
145  CHECK_STATUS(vpiSubmitConvertImageFormat(stream, VPI_BACKEND_CUDA, outputNV12, output, NULL));
146 
147  // Wait until the algorithm finishes processing
148  CHECK_STATUS(vpiStreamSync(stream));
149 
150  // Now let's retrieve the output image contents and output it to disk
151  {
152  // Lock output image to retrieve its data on cpu memory
153  VPIImageData outData;
154  CHECK_STATUS(vpiImageLockData(output, VPI_LOCK_READ, VPI_IMAGE_BUFFER_HOST_PITCH_LINEAR, &outData));
155 
156  // Returned data consists of host-accessible memory buffers in pitch-linear layout.
158 
159  VPIImageBufferPitchLinear &outDataPitch = outData.buffer.pitch;
160 
161  cv::Mat cvOut(outDataPitch.planes[0].height, outDataPitch.planes[0].width, CV_8UC3,
162  outDataPitch.planes[0].data, outDataPitch.planes[0].pitchBytes);
163  imwrite("scaled_" + strBackend + ".png", cvOut);
164 
165  // Done handling output image, don't forget to unlock it.
166  CHECK_STATUS(vpiImageUnlock(output));
167  }
168  }
169  catch (std::exception &e)
170  {
171  std::cerr << e.what() << std::endl;
172  retval = 1;
173  }
174 
175  // Clean up
176 
177  // Make sure stream is synchronized before destroying the objects
178  // that might still be in use.
179  vpiStreamSync(stream);
180 
181  vpiImageDestroy(image);
182  vpiImageDestroy(imageNV12);
183  vpiImageDestroy(output);
184  vpiStreamDestroy(stream);
185 
186  return retval;
187 }
Declares functions that handle image format conversion.
#define VPI_IMAGE_FORMAT_BGR8
Single plane with interleaved BGR 8-bit channel.
Definition: ImageFormat.h:324
#define VPI_IMAGE_FORMAT_NV12_ER
YUV420sp 8-bit pitch-linear format with full range.
Definition: ImageFormat.h:222
Functions and structures for dealing with VPI images.
Functions for handling OpenCV interoperability with VPI.
Declares functions that implement the Rescale algorithm.
Declaration of VPI status codes handling functions.
Declares functions dealing with VPI streams.
VPIStatus vpiSubmitConvertImageFormat(VPIStream stream, uint64_t backend, VPIImage input, VPIImage output, const VPIConvertImageFormatParams *params)
Converts the image contents to the desired format, with optional scaling and offset.
VPIImageBuffer buffer
Stores the image contents.
Definition: Image.h:241
VPIImagePlanePitchLinear planes[VPI_MAX_PLANE_COUNT]
Data of all image planes in pitch-linear layout.
Definition: Image.h:160
VPIImageBufferPitchLinear pitch
Image stored in pitch-linear layout.
Definition: Image.h:210
void * data
Pointer to the first row of this plane.
Definition: Image.h:141
VPIImageBufferType bufferType
Type of image buffer.
Definition: Image.h:238
int32_t height
Height of this plane in pixels.
Definition: Image.h:123
int32_t width
Width of this plane in pixels.
Definition: Image.h:119
int32_t pitchBytes
Difference in bytes of beginning of one row and the beginning of the previous.
Definition: Image.h:134
void vpiImageDestroy(VPIImage img)
Destroy an image instance.
struct VPIImageImpl * VPIImage
A handle to an image.
Definition: Types.h:256
VPIStatus vpiImageLockData(VPIImage img, VPILockMode mode, VPIImageBufferType bufType, VPIImageData *data)
Acquires the lock on an image object and returns the image contents.
VPIStatus vpiImageCreate(int32_t width, int32_t height, VPIImageFormat fmt, uint64_t flags, VPIImage *img)
Create an empty image instance with the specified flags.
VPIStatus vpiImageUnlock(VPIImage img)
Releases the lock on an image object.
@ VPI_IMAGE_BUFFER_HOST_PITCH_LINEAR
Host-accessible with planes in pitch-linear memory layout.
Definition: Image.h:172
Stores the image plane contents.
Definition: Image.h:150
Stores information about image characteristics and content.
Definition: Image.h:234
VPIStatus vpiImageCreateWrapperOpenCVMat(const cv::Mat &mat, VPIImageFormat fmt, uint64_t flags, VPIImage *img)
Wraps a cv::Mat in an VPIImage with the given image format.
VPIStatus vpiSubmitRescale(VPIStream stream, uint64_t backend, VPIImage input, VPIImage output, VPIInterpolationType interpolationType, VPIBorderExtension border, uint64_t flags)
Changes the size and scale of a 2D image.
struct VPIStreamImpl * VPIStream
A handle to a stream.
Definition: Types.h:250
VPIStatus vpiStreamSync(VPIStream stream)
Blocks the calling thread until all submitted commands in this stream queue are done (queue is empty)...
VPIBackend
VPI Backend types.
Definition: Types.h:91
void vpiStreamDestroy(VPIStream stream)
Destroy a stream instance and deallocate all HW resources.
VPIStatus vpiStreamCreate(uint64_t flags, VPIStream *stream)
Create a stream instance.
@ VPI_BACKEND_CUDA
CUDA backend.
Definition: Types.h:93
@ VPI_BACKEND_VIC
VIC backend.
Definition: Types.h:95
@ VPI_BACKEND_CPU
CPU backend.
Definition: Types.h:92
@ VPI_BORDER_CLAMP
Border pixels are repeated indefinitely.
Definition: Types.h:279
@ VPI_INTERP_LINEAR
Linear interpolation.
Definition: Interpolation.h:93
@ VPI_LOCK_READ
Lock memory only for reading.
Definition: Types.h:617