VPI - Vision Programming Interface

0.4.4 Release



The Convolution algorithm performs a 2D convolution operation on the input image with the provided 2D kernel. This is useful when the kernel isn't separable and its dimensions are smaller than 5x5. In other cases, it's usually preferable to use the Separable Convolution algorithm due to its speed.

Input Kernel Output

\[ \begin{bmatrix} 1 & 0 & -1 \\ 0 & 0 & 0 \\ -1 & 0 & 1 \end{bmatrix} \]


Discrete 2D convolution is implemented using the following discrete function:

\[ I'[x,y] = \sum_{m=0}^{k_h} \sum_{n=0}^{k_w} K[m,n] \times I[x-(n-\lfloor k_w/2 \rfloor), y-(m-\lfloor k_h/2 \rfloor) ] \]


  • \(I\) is the input image.
  • \(I'\) is the result image.
  • \(K\) is the convolution kernel.
  • \(k_w,k_h\) are the kernel's width and height, respectively.
Most computer vision libraries expect the kernel to be reversed before calling their convolution functions. Not so with VPI, we implement an actual convolution, not cross-correlation. Naturally, this is irrelevant if the kernel is symmetric.


  1. Initialization phase
    1. Include the header that defines the needed functions and structures.
    2. Define the input image object.
      VPIImage input = /*...*/;
    3. Create the output image. It gets its dimensions and format from the input image.
      uint32_t w, h;
      vpiImageGetSize(input, &w, &h);
      vpiImageGetType(input, &type);
      VPIImage output;
      vpiImageCreate(w, h, type, 0, &output);
    4. Create the stream where the algorithm will be submitted for execution.
      VPIStream stream;
      vpiStreamCreate(0, &stream);
  2. Processing phase
    1. Define the kernel to be used. In this case, a simple 3x3 edge detector.
      float kernel[3 * 3] = { 1, 0,-1,
      0, 0, 0,
      -1, 0, 1};
    2. Submit the algorithm to the stream, passing the kernel and other parameters. It'll be executed by the CPU backend.
      vpiSubmitConvolution(stream, VPI_BACKEND_CPU, input, output, kernel, 3, 3, VPI_BOUNDARY_COND_ZERO);
    3. Optionally, wait until the processing is done.
  3. Cleanup phase
    1. Free resources held by the stream and the input and output images.

Consult the Image Convolution for a complete example.

For more details, consult the Convolution API reference.

Limitations and Constraints

Constraints for specific backends supersede the ones specified for all backends.

All Backends


  • Only available on Jetson Xavier devices.
  • Input and output dimensions must be between 65x33 and 3264x2448.
  • Minimum convolution kernel size is 2x2.
  • Maximum convolution kernel size is 11x11.
  • Kernel weights are restricted to \(|weight| < 1\)
  • Only VPI_BOUNDARY_COND_ZERO is accepted.


  • Not implemented.


For information on how to use the performance table below, see Algorithm Performance Tables.
Before comparing measurements, consult Comparing Algorithm Elapsed Times.
For further information on how performance was benchmarked, see Performance Measurement.

VPIStatus vpiSubmitConvolution(VPIStream stream, VPIBackend backend, VPIImage input, VPIImage output, const float *kernelData, uint32_t kernelWidth, uint32_t kernelHeight, VPIBoundaryCond boundary)
Runs a generic 2D convolution over an image.
VPIStatus vpiStreamCreate(uint32_t flags, VPIStream *stream)
Create a stream instance.
Declares functions to perform image filtering with convolution kernels.
VPIStatus vpiStreamSync(VPIStream stream)
Blocks the calling thread until all submitted commands in this stream queue are done (queue is empty)...
struct VPIStreamImpl * VPIStream
A handle to a stream.
Definition: Types.h:190
void vpiStreamDestroy(VPIStream stream)
Destroy a stream instance and deallocate all HW resources.
VPIStatus vpiImageCreate(uint32_t width, uint32_t height, VPIImageFormat fmt, uint32_t flags, VPIImage *img)
Create an empty image instance with the specified flags.
void vpiImageDestroy(VPIImage img)
Destroy an image instance.
struct VPIImageImpl * VPIImage
A handle to an image.
Definition: Types.h:196
VPIStatus vpiImageGetSize(VPIImage img, uint32_t *width, uint32_t *height)
Get the image size in pixels.
All pixels outside the image are considered to be zero.
Definition: Types.h:218
CPU backend.
Definition: Types.h:90
Pre-defined image formats.
Definition: ImageFormat.h:94
VPIStatus vpiImageGetType(VPIImage img, VPIImageFormat *type)
Get the image format.