Image Resampler

Overview

The Image Resampler algorithm is used to rescale the input image by means of resampling its content to make it conform to the output image dimensions.

No pre-filtering is applied, it's expected that the input content doesn't have frequencies higher than the Nyquist limit to avoid aliasing artifacts when downsampling.

Several interpolation methods are available, allowing trade-offs between quality and performance.

Input	Factor	Output
	\begin{align*} f_x &= 2/3 \\[5pt] f_y &= 3/2 \end{align*}

Implementation

For every output pixel, calculate the corresponding input pixel using the formula:

\[ \mathit{out}[x,y] = P(f_x x, f_y y) \]

where P depends on the interpolation method used.

Sampling operation considers that whole coordinates fall on pixel center.

Usage

1. Initialization phase
   1. Include the header that defines the needed functions and structures.

      #include <vpi/algo/ImageResampler.h>

   2. Define the stream on which the algorithm will be executed and the input image.

          VPIStream stream = /*...*/;
          VPIImage input   = /*...*/;

   3. Create an output image with the required size and matching image type.

          uint32_t w, h;
          vpiImageGetSize(input, &w, &h);
       
          VPIImageType type;
          vpiImageGetType(input, &type);
       
          VPIImage output;
          vpiImageCreate(w * 2 / 3.0f, h * 3 / 2.0f, type, 0, &output);

2. Processing phase
   1. Submit the algorithm to the stream along with all parameters.

          vpiSubmitImageResampler(stream, input, output, VPI_INTERP_LINEAR_FAST, VPI_BOUNDARY_COND_ZERO);

   2. Optionally, wait until the processing is done.

          vpiStreamSync(stream);

Consult the Image Resample for a complete example.

For more details, consult the API reference.

Limitations and Constraints

Constraints for specific backends supersede the ones specified for all backends.

All Backends

• Input and output images must have the same dimensions and type.
• The following interpolation methods are supported:
  • VPI_INTERP_NEAREST
  • VPI_INTERP_LINEAR
  • VPI_INTERP_LINEAR_PRECISE
  • VPI_INTERP_CATMULL_ROM
  • VPI_INTERP_CATMULL_ROM_PRECISE
• The following image types are accepted:
  • VPI_IMAGE_TYPE_U8
  • VPI_IMAGE_TYPE_S8
  • VPI_IMAGE_TYPE_U16
  • VPI_IMAGE_TYPE_S16
• The following boundary conditions are accepted.
  • VPI_BOUNDARY_COND_ZERO
  • VPI_BOUNDARY_COND_CLAMP
• Minimum convolution kernel size is 1x1, maximum is 11x11.

PVA

• Not implemented

Performance

For further information on how performance was benchmarked, see Performance Measurement.

Jetson AGX Xavier

input	output	type	interp.	CPU	CUDA	PVA
640x480	1920x1080	u8	nearest	0.410 ms	0.0730 ms	n/a
640x480	1920x1080	u8	linear	0.94 ms	0.0737 ms	n/a
640x480	1920x1080	u8	catmull	0.95 ms	0.2193 ms	n/a
960x540	1920x1080	u8	nearest	0.435 ms	0.0757 ms	n/a
960x540	1920x1080	u8	linear	1.00 ms	0.0745 ms	n/a
960x540	1920x1080	u8	catmull	1.03 ms	0.2235 ms	n/a
1920x1080	640x480	u8	nearest	0.202 ms	0.0201 ms	n/a
1920x1080	640x480	u8	linear	0.575 ms	0.0298 ms	n/a
1920x1080	640x480	u8	catmull	0.704 ms	0.0467 ms	n/a
1280x960	640x480	u8	nearest	0.196 ms	0.0201 ms	n/a
1280x960	640x480	u8	linear	0.155 ms	0.0173 ms	n/a
1280x960	640x480	u8	catmull	0.638 ms	0.0432 ms	n/a
640x480	1920x1080	u16	nearest	0.51 ms	0.0926 ms	n/a
640x480	1920x1080	u16	linear	0.67 ms	0.0928 ms	n/a
640x480	1920x1080	u16	catmull	0.9 ms	0.2339 ms	n/a
960x540	1920x1080	u16	nearest	0.529 ms	0.0931 ms	n/a
960x540	1920x1080	u16	linear	1.06 ms	0.0914 ms	n/a
960x540	1920x1080	u16	catmull	0.98 ms	0.2394 ms	n/a
1920x1080	640x480	u16	nearest	0.234 ms	0.0321 ms	n/a
1920x1080	640x480	u16	linear	0.59 ms	0.0538 ms	n/a
1920x1080	640x480	u16	catmull	0.625 ms	0.0667 ms	n/a
1280x960	640x480	u16	nearest	0.219 ms	0.0251 ms	n/a
1280x960	640x480	u16	linear	0.208 ms	0.03037 ms	n/a
1280x960	640x480	u16	catmull	0.570 ms	0.0528 ms	n/a

Jetson TX2

input	output	type	interp.	CPU	CUDA	PVA
640x480	1920x1080	u8	nearest	0.43 ms	0.1996 ms	n/a
640x480	1920x1080	u8	linear	1.058 ms	0.1924 ms	n/a
640x480	1920x1080	u8	catmull	2.44 ms	0.555 ms	n/a
960x540	1920x1080	u8	nearest	0.559 ms	0.1999 ms	n/a
960x540	1920x1080	u8	linear	1.19 ms	0.195 ms	n/a
960x540	1920x1080	u8	catmull	2.566 ms	0.556 ms	n/a
1920x1080	640x480	u8	nearest	0.267 ms	0.0480 ms	n/a
1920x1080	640x480	u8	linear	0.951 ms	0.0593 ms	n/a
1920x1080	640x480	u8	catmull	2.99 ms	0.1058 ms	n/a
1280x960	640x480	u8	nearest	0.228 ms	0.0482 ms	n/a
1280x960	640x480	u8	linear	0.102 ms	0.0375 ms	n/a
1280x960	640x480	u8	catmull	2.14 ms	0.0966 ms	n/a
640x480	1920x1080	u16	nearest	0.616 ms	0.215 ms	n/a
640x480	1920x1080	u16	linear	1.171 ms	0.215 ms	n/a
640x480	1920x1080	u16	catmull	2.1 ms	0.552 ms	n/a
960x540	1920x1080	u16	nearest	0.746 ms	0.221 ms	n/a
960x540	1920x1080	u16	linear	1.299 ms	0.219 ms	n/a
960x540	1920x1080	u16	catmull	2.96 ms	0.559 ms	n/a
1920x1080	640x480	u16	nearest	0.445 ms	0.0796 ms	n/a
1920x1080	640x480	u16	linear	1.055 ms	0.114 ms	n/a
1920x1080	640x480	u16	catmull	2.50 ms	0.183 ms	n/a
1280x960	640x480	u16	nearest	0.361 ms	0.0536 ms	n/a
1280x960	640x480	u16	linear	0.347 ms	0.0716 ms	n/a
1280x960	640x480	u16	catmull	2.54 ms	0.127 ms	n/a

Jetson Nano

input	output	type	interp.	CPU	CUDA	PVA
640x480	1920x1080	u8	nearest	0.799 ms	0.5540 ms	n/a
640x480	1920x1080	u8	linear	2.106 ms	0.5289 ms	n/a
640x480	1920x1080	u8	catmull	3.92 ms	1.5413 ms	n/a
960x540	1920x1080	u8	nearest	0.941 ms	0.5541 ms	n/a
960x540	1920x1080	u8	linear	2.274 ms	0.5259 ms	n/a
960x540	1920x1080	u8	catmull	4.26 ms	1.545 ms	n/a
1920x1080	640x480	u8	nearest	0.279 ms	0.0981 ms	n/a
1920x1080	640x480	u8	linear	1.234 ms	0.1188 ms	n/a
1920x1080	640x480	u8	catmull	2.88 ms	0.2638 ms	n/a
1280x960	640x480	u8	nearest	0.2705 ms	0.0910 ms	n/a
1280x960	640x480	u8	linear	0.107 ms	0.0795 ms	n/a
1280x960	640x480	u8	catmull	2.296 ms	0.2506 ms	n/a
640x480	1920x1080	u16	nearest	0.999 ms	0.5518 ms	n/a
640x480	1920x1080	u16	linear	2.111 ms	0.532 ms	n/a
640x480	1920x1080	u16	catmull	3.350 ms	1.522 ms	n/a
960x540	1920x1080	u16	nearest	1.106 ms	0.552 ms	n/a
960x540	1920x1080	u16	linear	2.334 ms	0.533 ms	n/a
960x540	1920x1080	u16	catmull	3.56 ms	1.5287 ms	n/a
1920x1080	640x480	u16	nearest	0.431 ms	0.1429 ms	n/a
1920x1080	640x480	u16	linear	1.313 ms	0.224 ms	n/a
1920x1080	640x480	u16	catmull	2.03 ms	0.362 ms	n/a
1280x960	640x480	u16	nearest	0.353 ms	0.1103 ms	n/a
1280x960	640x480	u16	linear	0.333 ms	0.1535 ms	n/a
1280x960	640x480	u16	catmull	2.038 ms	0.2907 ms	n/a

References

• Daniel Ruijters, Bart M. ter Romeny, Paul Suetens (2008) "Efficient GPU-Based Texture Interpolation using Uniform B-Spline"
  Journal of Graphics Tools, 13:4 61-69.