Overview

A Bilateral Image Filter is a non-linear, edge-preserving smoothing filter that is commonly used in Computer Vision as a simple noise-reduction stage in a pipeline. It calculates the intensity of each output pixel as a weighted average of intensity values from nearby pixels in the input image. Crucially, the weights depend not only on the Euclidean distance between current and neighbor pixels, but also on the radiometric differences (e.g., color intensity differences) between them. The outcome is that edges are preserved while regions with similar intensities are smoothed out.

Input	Parameters	Output
	\begin{align} \mathit{kernelSize} &= 7x7 \\ \sigma_s &= 1.7 \\ \sigma_r &= 50 \end{align}

Implementation

The bilateral image filter is defined as:

\[ I'(p) = \frac{1}{W_p} \sum_{q\in\Omega}I(p)k_r(\|I(q) - I(p)\|)k_s(\|p-q\|) \]

and the normalization term, W, is defined as:

\[ W_p = \sum_{q\in\Omega}k_r(\|I(p)-I(q)\|)k_s(\|p-q\|) \]

where

\(I\) and \(I'\) are the input and output images, respectively.
\(k_r\) and \(k_s\) are the range and space kernels, respectively, defined as the following non-normalized Gaussian functions:
\begin{align*} k_r(p) &= e^{-\frac{\|p\|^2}{2\sigma_r^2}} \\ k_s(p) &= e^{-\frac{\|p\|^2}{2\sigma_s^2}} \end{align*}
\(\sigma_r\) controls the intensity range that is smoothed out. Higher values will lead to larger regions being smoothed out. The \(\sigma_r\) value should be selected with the dynamic range of the image pixel values in mind.
\(\sigma_s\) controls smoothing factor. Higher values will lead to more smoothing.

Usage

Initialization phase
1. Include the header that defines the Gaussian filter function.
  #include <vpi/algo/BilateralImageFilter.h>
2. Define the stream on which the algorithm will be executed, the input and output images.
  VPIStream stream = /*...*/;
  
  VPIImage input = /*...*/;
3. Create the output image.
  uint32_t w, h;
  
  vpiImageGetSize(input, &w, &h);
  
  VPIImageType type;
  
  vpiImageGetType(input, &type);
  
  VPIImage output;
  
  vpiImageCreate(w, h, type, 0, &output);
Processing phase
1. Submit the algorithm to the stream, along with all parameters. Here we're using a 7x7 kernel with \(\sigma_r=50\) and \(\sigma_s=1.7\).
  vpiSubmitBilateralImageFilter(stream, input, output, 7, 50, 1.7, VPI_BOUNDARY_COND_ZERO);
2. Optionally, wait until the processing is done.
  vpiStreamSync(stream);

Limitations and Constraints

Constraints for specific backends supersede the ones specified for all backends.

All Backends

The following image types are accepted:
The following boundary conditions are accepted:
- VPI_BOUNDARY_COND_ZERO
- VPI_BOUNDARY_COND_CLAMP

PVA

Not implemented

Performance

For further information on how performance was benchmarked, see Performance Measurement.

Jetson AGX Xavier
size	type	kernel	CPU	CUDA	PVA
1920x1080	u8	3x3	1.51 ms	0.1820 ms	n/a
1920x1080	u8	5x5	3.57 ms	0.2598 ms	n/a
1920x1080	u8	7x7	9.8 ms	0.5482 ms	n/a
1920x1080	u8	11x11	24.53 ms	1.3467 ms	n/a
1920x1080	u16	3x3	1.87 ms	0.1907 ms	n/a
1920x1080	u16	5x5	4.05 ms	0.2644 ms	n/a
1920x1080	u16	7x7	11.0 ms	0.5485 ms	n/a
1920x1080	u16	11x11	27.35 ms	1.3463 ms	n/a

Jetson TX2
size	type	kernel	CPU	CUDA	PVA
1920x1080	u8	3x3	5.5 ms	0.644 ms	n/a
1920x1080	u8	5x5	13.4 ms	0.905 ms	n/a
1920x1080	u8	7x7	38.3 ms	1.9385 ms	n/a
1920x1080	u8	11x11	65.0 ms	4.4045 ms	n/a
1920x1080	u16	3x3	5.53 ms	0.683 ms	n/a
1920x1080	u16	5x5	17.08 ms	0.928 ms	n/a
1920x1080	u16	7x7	39 ms	1.9158 ms	n/a
1920x1080	u16	11x11	82.2 ms	4.3486 ms	n/a

Jetson Nano
size	type	kernel	CPU	CUDA	PVA
1920x1080	u8	3x3	13.63 ms	1.674 ms	n/a
1920x1080	u8	5x5	34.68 ms	2.456 ms	n/a
1920x1080	u8	7x7	80.8 ms	5.5499 ms	n/a
1920x1080	u8	11x11	179.7 ms	12.561 ms	n/a
1920x1080	u16	3x3	17.54 ms	1.709 ms	n/a
1920x1080	u16	5x5	44.9 ms	2.485 ms	n/a
1920x1080	u16	7x7	90.5 ms	5.4695 ms	n/a
1920x1080	u16	11x11	203.5 ms	12.428 ms	n/a

VPI - Vision Programming Interface