Image Resampler

Overview

The Image Resampler algorithm is used to rescale the input image by means of resampling its content to make it conform to the output image dimensions.

No pre-filtering is applied, it's expected that the input content doesn't have frequencies higher than the Nyquist limit to avoid aliasing artifacts when downsampling.

Several interpolation methods are available, allowing trade-offs between quality and performance.

Input	Factor	Output
	\begin{align*} f_x &= 2/3 \\[5pt] f_y &= 3/2 \end{align*}

Implementation

For every output pixel, calculate the corresponding input pixel using the formula:

\[ \mathit{out}[x,y] = P(f_x x, f_y y) \]

where P depends on the interpolation method used.

Sampling operation considers that whole coordinates fall on pixel center.

Usage

1. Initialization phase
   1. Include the header that defines the needed functions and structures.

      #include <vpi/algo/ImageResampler.h>

   2. Define the stream on which the algorithm will be executed and the input image.

          VPIStream stream = /*...*/;
          VPIImage input   = /*...*/;

   3. Create an output image with the required size and matching image type.

          uint32_t w, h;
          vpiImageGetSize(input, &w, &h);
       
          VPIImageType type;
          vpiImageGetType(input, &type);
       
          VPIImage output;
          vpiImageCreate(w * 2 / 3.0f, h * 3 / 2.0f, type, 0, &output);

2. Processing phase
   1. Submit the algorithm to the stream along with all parameters.

          vpiSubmitImageResampler(stream, input, output, VPI_INTERP_LINEAR_FAST, VPI_BOUNDARY_COND_ZERO);

   2. Optionally, wait until the processing is done.

          vpiStreamSync(stream);

Consult the Image Resample for a complete example.

Limitations and Constraints

Constraints for specific backends supersede the ones specified for all backends.

All Backends

• Input and output images must have the same dimensions and type.
• The following interpolation methods are supported:
  • VPI_INTERP_NEAREST
  • VPI_INTERP_LINEAR
  • VPI_INTERP_LINEAR_PRECISE
  • VPI_INTERP_CATMULL_ROM
  • VPI_INTERP_CATMULL_ROM_PRECISE
• The following image types are accepted:
  • VPI_IMAGE_TYPE_U8
  • VPI_IMAGE_TYPE_S8
  • VPI_IMAGE_TYPE_U16
  • VPI_IMAGE_TYPE_S16
• The following boundary conditions are accepted.
  • VPI_BOUNDARY_COND_ZERO
  • VPI_BOUNDARY_COND_CLAMP
• Minimum convolution kernel size is 1x1, maximum is 11x11.

PVA

• Not implemented

Performance

For further information on how performance was benchmarked, see Performance Measurement.

Jetson AGX Xavier

input	output	type	interp.	CPU	CUDA	PVA
640x480	1920x1080	u8	nearest	0.407 ms	0.0760 ms	n/a
640x480	1920x1080	u8	linear	0.92 ms	0.0751 ms	n/a
640x480	1920x1080	u8	catmull	0.99 ms	0.2217 ms	n/a
960x540	1920x1080	u8	nearest	0.43 ms	0.0776 ms	n/a
960x540	1920x1080	u8	linear	0.96 ms	0.0759 ms	n/a
960x540	1920x1080	u8	catmull	1.03 ms	0.2264 ms	n/a
1920x1080	640x480	u8	nearest	0.199 ms	0.0495 ms	n/a
1920x1080	640x480	u8	linear	0.58 ms	0.0483 ms	n/a
1920x1080	640x480	u8	catmull	0.69 ms	0.0486 ms	n/a
1280x960	640x480	u8	nearest	0.190 ms	0.0480 ms	n/a
1280x960	640x480	u8	linear	0.151 ms	0.039 ms	n/a
1280x960	640x480	u8	catmull	0.624 ms	0.0474 ms	n/a
640x480	1920x1080	u16	nearest	0.51 ms	0.0937 ms	n/a
640x480	1920x1080	u16	linear	1.0 ms	0.0936 ms	n/a
640x480	1920x1080	u16	catmull	1.21 ms	0.2389 ms	n/a
960x540	1920x1080	u16	nearest	0.596 ms	0.0948 ms	n/a
960x540	1920x1080	u16	linear	1.10 ms	0.0942 ms	n/a
960x540	1920x1080	u16	catmull	1.1 ms	0.2431 ms	n/a
1920x1080	640x480	u16	nearest	0.25 ms	0.0465 ms	n/a
1920x1080	640x480	u16	linear	0.584 ms	0.0535 ms	n/a
1920x1080	640x480	u16	catmull	0.67 ms	0.0665 ms	n/a
1280x960	640x480	u16	nearest	0.234 ms	0.0464 ms	n/a
1280x960	640x480	u16	linear	0.206 ms	0.0385 ms	n/a
1280x960	640x480	u16	catmull	0.581 ms	0.0531 ms	n/a

Jetson TX2

input	output	type	interp.	CPU	CUDA	PVA
640x480	1920x1080	u8	nearest	0.430 ms	0.1986 ms	n/a
640x480	1920x1080	u8	linear	1.061 ms	0.191 ms	n/a
640x480	1920x1080	u8	catmull	2.51 ms	0.554 ms	n/a
960x540	1920x1080	u8	nearest	0.495 ms	0.199 ms	n/a
960x540	1920x1080	u8	linear	1.151 ms	0.193 ms	n/a
960x540	1920x1080	u8	catmull	2.587 ms	0.557 ms	n/a
1920x1080	640x480	u8	nearest	0.236 ms	0.128 ms	n/a
1920x1080	640x480	u8	linear	0.954 ms	0.1266 ms	n/a
1920x1080	640x480	u8	catmull	2.92 ms	0.127 ms	n/a
1280x960	640x480	u8	nearest	0.231 ms	0.1280 ms	n/a
1280x960	640x480	u8	linear	0.093 ms	0.1086 ms	n/a
1280x960	640x480	u8	catmull	2.11 ms	0.1263 ms	n/a
640x480	1920x1080	u16	nearest	0.619 ms	0.213 ms	n/a
640x480	1920x1080	u16	linear	1.182 ms	0.215 ms	n/a
640x480	1920x1080	u16	catmull	2.11 ms	0.554 ms	n/a
960x540	1920x1080	u16	nearest	0.722 ms	0.218 ms	n/a
960x540	1920x1080	u16	linear	1.30 ms	0.216 ms	n/a
960x540	1920x1080	u16	catmull	2.38 ms	0.560 ms	n/a
1920x1080	640x480	u16	nearest	0.420 ms	0.1251 ms	n/a
1920x1080	640x480	u16	linear	1.040 ms	0.1279 ms	n/a
1920x1080	640x480	u16	catmull	2.47 ms	0.180 ms	n/a
1280x960	640x480	u16	nearest	0.371 ms	0.1277 ms	n/a
1280x960	640x480	u16	linear	0.353 ms	0.108 ms	n/a
1280x960	640x480	u16	catmull	2.43 ms	0.1286 ms	n/a

Jetson Nano

input	output	type	interp.	CPU	CUDA	PVA
640x480	1920x1080	u8	nearest	0.778 ms	0.5543 ms	n/a
640x480	1920x1080	u8	linear	2.092 ms	0.5288 ms	n/a
640x480	1920x1080	u8	catmull	3.953 ms	1.542 ms	n/a
960x540	1920x1080	u8	nearest	0.930 ms	0.5542 ms	n/a
960x540	1920x1080	u8	linear	2.287 ms	0.5265 ms	n/a
960x540	1920x1080	u8	catmull	4.35 ms	1.5471 ms	n/a
1920x1080	640x480	u8	nearest	0.2712 ms	0.142 ms	n/a
1920x1080	640x480	u8	linear	1.219 ms	0.145 ms	n/a
1920x1080	640x480	u8	catmull	2.89 ms	0.2616 ms	n/a
1280x960	640x480	u8	nearest	0.2683 ms	0.143 ms	n/a
1280x960	640x480	u8	linear	0.1034 ms	0.113 ms	n/a
1280x960	640x480	u8	catmull	2.230 ms	0.2487 ms	n/a
640x480	1920x1080	u16	nearest	0.981 ms	0.5520 ms	n/a
640x480	1920x1080	u16	linear	2.144 ms	0.530 ms	n/a
640x480	1920x1080	u16	catmull	3.232 ms	1.522 ms	n/a
960x540	1920x1080	u16	nearest	1.093 ms	0.551 ms	n/a
960x540	1920x1080	u16	linear	2.299 ms	0.533 ms	n/a
960x540	1920x1080	u16	catmull	3.51 ms	1.528 ms	n/a
1920x1080	640x480	u16	nearest	0.4167 ms	0.146 ms	n/a
1920x1080	640x480	u16	linear	1.302 ms	0.2222 ms	n/a
1920x1080	640x480	u16	catmull	2.07 ms	0.360 ms	n/a
1280x960	640x480	u16	nearest	0.355 ms	0.143 ms	n/a
1280x960	640x480	u16	linear	0.3592 ms	0.1503 ms	n/a
1280x960	640x480	u16	catmull	1.930 ms	0.2879 ms	n/a

References

• Daniel Ruijters, Bart M. ter Romeny, Paul Suetens (2008) "Efficient GPU-Based Texture Interpolation using Uniform B-Spline"
  Journal of Graphics Tools, 13:4 61-69.