## VPI - Vision Programming Interface

#### 0.4.4 Release

Separable Convolution

# Overview

The Separable Convolution algorithm performs a 2D convolution operation, but takes advantage of the fact that the 2D kernel is separable. The user passes one horizontal and one vertical 1D kernel. This usually leads to better performance, especially for kernels larger than 5x5. For smaller kernels, it's preferable to use Convolution algorithm with a 2D kernel directly.

Input Sobel kernel Output

\begin{eqnarray*} k_{col} &=& \frac{1}{64} \begin{bmatrix} 1 \\ 6 \\ 15 \\ 20 \\ 15 \\ 6 \\ 1 \end{bmatrix} \\ k_{row} &=& \begin{bmatrix} -1 & -5 & -6 & 0 & 6 & 5 & 1 \end{bmatrix} \end{eqnarray*}

# Implementation

Discrete 2D convolution is implemented using the following discrete function:

\begin{eqnarray*} I'[x,y] &=& \sum_{m=0}^{k_w} K_{row}[m] \times I[x,y-(m - \lfloor k_w/2 \rfloor)] \\ I''[x,y] &=& \sum_{m=0}^{k_h} K_{col}[m] \times I'[x-(m - \lfloor k_h/2 \rfloor),y] \end{eqnarray*}

Where:

• $$I$$ is the input image.
• $$I'$$ is the temporary image with convolution along the rows.
• $$I''$$ is the final result.
• $$K_{row}$$ is the row convolution kernel.
• $$K_{col}$$ is the column convolution kernel.
• $$k_w,k_h$$ are the kernel's width and height, respectively.
Note
Most computer vision libraries expect the kernel to be reversed before calling their convolution functions. Not so with VPI, we implement a actual convolution, not cross-correlation. Naturally, this is irrelevant if the kernel is symmetric.

# Usage

1. Initialization phase
1. Include the header that defines the needed functions and structures.
2. Define the input image object.
VPIImage input = /*...*/;
3. Create the output image. It gets its dimensions and format from the input image.
uint32_t w, h;
vpiImageGetSize(input, &w, &h);
vpiImageGetType(input, &type);
VPIImage output;
vpiImageCreate(w, h, type, 0, &output);
4. Create the stream where the algorithm will be submitted for execution.
VPIStream stream;
vpiStreamCreate(0, &stream);
2. Processing phase
1. Define the kernel to be used. In this case, a simple 7x7 Sobel filter.
float sobel_row[7] = {-1, -5, -6, 0, +6, +5, +1};
float sobel_col[7] = {1/64.f, 6/64.f, 15/64.f, 20/64.f, 15/64.f, 6/64.f, 1/64.f};
2. Submit the algorithm to the stream, passing the 1D kernels and remaining arguments. I'll be executed by the CUDA backend.
vpiSubmitSeparableConvolution(stream, VPI_BACKEND_CUDA, input, output, sobel_row, 7, sobel_col, 7, VPI_BOUNDARY_COND_ZERO);
3. Optionally, wait until the processing is done.
vpiStreamSync(stream);
3. Cleanup phase
1. Free resources held by the stream and the input and output images.

For more details, consult the Convolution API reference.

# Limitations and Constraints

Constraints for specific backends supersede the ones specified for all backends.

## All Backends

• Input and output images must have the same dimensions and type.
• The following image formats are accepted:
• Minimum 1D convolution kernel size is 1, maximum is 11.
• The following boundary conditions are accepted.

## PVA

• Only available on Jetson Xavier devices.
• Input and output dimensions must be between 160x92 and 3264x2448.
• Minimum 1D convolution kernel size is 2, maximum is 11.
• Horizontal and vertical kernel sizes must be equal, i.e., only square kernels can be used.
• Kernel weights are restricted to $$|weight| < 1$$.
• The following image formats are the only ones accepted:
• The following boundary conditions are accepted.

## VIC

• Not implemented.

# Performance

For information on how to use the performance table below, see Algorithm Performance Tables.
Before comparing measurements, consult Comparing Algorithm Elapsed Times.
For further information on how performance was benchmarked, see Performance Measurement.

-
vpiStreamCreate
VPIStatus vpiStreamCreate(uint32_t flags, VPIStream *stream)
Create a stream instance.
Convolution.h
Declares functions to perform image filtering with convolution kernels.
vpiStreamSync
VPIStatus vpiStreamSync(VPIStream stream)
Blocks the calling thread until all submitted commands in this stream queue are done (queue is empty)...
VPI_BACKEND_CUDA
@ VPI_BACKEND_CUDA
CUDA backend.
Definition: Types.h:91
VPIStream
struct VPIStreamImpl * VPIStream
A handle to a stream.
Definition: Types.h:190
vpiStreamDestroy
void vpiStreamDestroy(VPIStream stream)
Destroy a stream instance and deallocate all HW resources.
vpiImageCreate
VPIStatus vpiImageCreate(uint32_t width, uint32_t height, VPIImageFormat fmt, uint32_t flags, VPIImage *img)
Create an empty image instance with the specified flags.
vpiImageDestroy
void vpiImageDestroy(VPIImage img)
Destroy an image instance.
VPIImage
struct VPIImageImpl * VPIImage
A handle to an image.
Definition: Types.h:196
vpiImageGetSize
VPIStatus vpiImageGetSize(VPIImage img, uint32_t *width, uint32_t *height)
Get the image size in pixels.
VPI_BOUNDARY_COND_ZERO
@ VPI_BOUNDARY_COND_ZERO
All pixels outside the image are considered to be zero.
Definition: Types.h:218
VPIImageFormat
VPIImageFormat
Pre-defined image formats.
Definition: ImageFormat.h:94
vpiSubmitSeparableConvolution
VPIStatus vpiSubmitSeparableConvolution(VPIStream stream, VPIBackend backend, VPIImage input, VPIImage output, const float *kernelXData, uint32_t kernelXSize, const float *kernelYData, uint32_t kernelYSize, VPIBoundaryCond boundary)
Runs a generic 2D convolution operation over an image, optimized for separable kernels.
vpiImageGetType
VPIStatus vpiImageGetType(VPIImage img, VPIImageFormat *type)
Get the image format.