Overview

A Bilateral Image Filter is a non-linear, edge-preserving smoothing filter that is commonly used in Computer Vision as a simple noise-reduction stage in a pipeline. It calculates the intensity of each output pixel as a weighted average of intensity values from nearby pixels in the input image. Crucially, the weights depend not only on the Euclidean distance between current and neighbor pixels, but also on the radiometric differences (e.g., color intensity differences) between them. The outcome is that edges are preserved while regions with similar intensities are smoothed out.

Input	Parameters	Output
	\begin{align} \mathit{kernelSize} &= 7x7 \\ \sigma_s &= 1.7 \\ \sigma_r &= 50 \end{align}

Implementation

The bilateral image filter is defined as:

\[ I'(p) = \frac{1}{W_p} \sum_{q\in\Omega}I(p)k_r(\|I(q) - I(p)\|)k_s(\|p-q\|) \]

and the normalization term, W, is defined as:

\[ W_p = \sum_{q\in\Omega}k_r(\|I(p)-I(q)\|)k_s(\|p-q\|) \]

where

\(I\) and \(I'\) are the input and output images, respectively.
\(k_r\) and \(k_s\) are the range and space kernels, respectively, defined as the following non-normalized Gaussian functions:
\begin{align*} k_r(p) &= e^{-\frac{\|p\|^2}{2\sigma_r^2}} \\ k_s(p) &= e^{-\frac{\|p\|^2}{2\sigma_s^2}} \end{align*}
\(\sigma_r\) controls the intensity range that is smoothed out. Higher values will lead to larger regions being smoothed out. The \(\sigma_r\) value should be selected with the dynamic range of the image pixel values in mind.
\(\sigma_s\) controls smoothing factor. Higher values will lead to more smoothing.

Usage

Initialization phase
1. Include the header that defines the Gaussian filter function.
  #include <vpi/algo/BilateralImageFilter.h>
2. Define the stream on which the algorithm will be executed, the input and output images.
  VPIStream stream = /*...*/;
  
  VPIImage input = /*...*/;
3. Create the output image.
  uint32_t w, h;
  
  vpiImageGetSize(input, &w, &h);
  
  VPIImageType type;
  
  vpiImageGetType(input, &type);
  
  VPIImage output;
  
  vpiImageCreate(w, h, type, 0, &output);
Processing phase
1. Submit the algorithm to the stream, along with all parameters. Here we're using a 7x7 kernel with \(\sigma_r=50\) and \(\sigma_s=1.7\).
  vpiSubmitBilateralImageFilter(stream, input, output, 7, 50, 1.7, VPI_BOUNDARY_COND_ZERO);
2. Optionally, wait until the processing is done.
  vpiStreamSync(stream);

For more details, consult the API reference.

Limitations and Constraints

Constraints for specific backends supersede the ones specified for all backends.

All Backends

The following image types are accepted:
The following boundary conditions are accepted:
- VPI_BOUNDARY_COND_ZERO
- VPI_BOUNDARY_COND_CLAMP

CPU

Does not support VPI_IMAGE_TYPE_NV12 inputs.

PVA

Not implemented

Performance

For further information on how performance was benchmarked, see Performance Measurement.

Jetson AGX Xavier
size	type	kernel	CPU	CUDA	PVA
1920x1080	u8	3x3	1.47 ms	0.1172 ms	n/a
1920x1080	u8	5x5	3.55 ms	0.2148 ms	n/a
1920x1080	u8	7x7	9.9 ms	0.4779 ms	n/a
1920x1080	u8	11x11	24.44 ms	1.4138 ms	n/a
1920x1080	u16	3x3	1.70 ms	0.1213 ms	n/a
1920x1080	u16	5x5	4.08 ms	0.2143 ms	n/a
1920x1080	u16	7x7	11.5 ms	0.4742 ms	n/a
1920x1080	u16	11x11	26.8 ms	1.4069 ms	n/a

Jetson TX2
size	type	kernel	CPU	CUDA	PVA
1920x1080	u8	3x3	4.89 ms	0.405 ms	n/a
1920x1080	u8	5x5	13.7 ms	0.6725 ms	n/a
1920x1080	u8	7x7	34.3 ms	1.317 ms	n/a
1920x1080	u8	11x11	68.0 ms	3.39 ms	n/a
1920x1080	u16	3x3	7.38 ms	0.430 ms	n/a
1920x1080	u16	5x5	16.40 ms	0.663 ms	n/a
1920x1080	u16	7x7	38.98 ms	1.2988 ms	n/a
1920x1080	u16	11x11	73.5 ms	3.367 ms	n/a

Jetson Nano
size	type	kernel	CPU	CUDA	PVA
1920x1080	u8	3x3	13.576 ms	1.0990 ms	n/a
1920x1080	u8	5x5	34.7 ms	1.8768 ms	n/a
1920x1080	u8	7x7	79.9 ms	3.7300 ms	n/a
1920x1080	u8	11x11	180.5 ms	9.734 ms	n/a
1920x1080	u16	3x3	17.63 ms	1.126 ms	n/a
1920x1080	u16	5x5	44.7 ms	1.8322 ms	n/a
1920x1080	u16	7x7	86.6 ms	3.6758 ms	n/a
1920x1080	u16	11x11	203.6 ms	9.690 ms	n/a

VPI - Vision Programming Interface

0.3.7 Release