NMS (Non-Maximum Suppression)#

Overview#

The Non-Maximum Suppression (NMS) operator operates on a single-channel int32_t image, typically a detector response map. For every pixel an \(N \times N\) window centered on that pixel is searched; if the pixel is the maximum of the window it is copied to the output, otherwise the output pixel is set to zero. Ties are resolved by position: among equal maxima within the window, the pixel that appears last in raster order (left to right, top to bottom) wins, that is, the one with the largest row index and, if those are equal, the largest column index. A rectangular plateau of equal values therefore leaves exactly one surviving pixel, at its bottom-right corner, rather than keeping all of them. The current implementation supports 3x3 and 5x5 windows.

Input (corner response, log scale)

Output (surviving maxima)

../../_images/in15.png
../../_images/out13.png

Nms Parameters: 5×5 window, constant border (value 0). The input is a Harris corner response; red dots are survivors above 5% of the maximum.

Algorithm Description#

\[\begin{split}dst[x, y] = \begin{cases} src[x, y] & \text{if } src[x, y] \ge src[u, v] \text{ for every earlier } (u, v) \text{, and } src[x, y] > src[u, v] \text{ for every later } (u, v) \\ 0 & \text{otherwise} \end{cases}\end{split}\]

Here \((u, v)\) ranges over the \(N \times N\) window centered on \((x, y)\), and earlier and later are in raster order within that window. The inequality against later taps is strict, which is what collapses a rectangular plateau of equal values to a single surviving pixel at its bottom-right corner instead of keeping every pixel of the plateau.

The output image has the same dimensions as the input. Pixels near the image border read their out-of-image neighbors through the configured border mode, either NVCV_BORDER_CONSTANT with a caller-supplied value or NVCV_BORDER_REPLICATE.

Implementation Details#

Dataflow Configuration#

Two RasterDataFlow streams move data between DRAM and VMEM: one reads the source with a halo of nmsSize / 2 pixels in each direction, the other writes the destination. The halo is what supplies the border pixels, so the kernel has no per-pixel boundary branches: PAD_CONST with the caller’s border value for NVCV_BORDER_CONSTANT, PAD_BPE for NVCV_BORDER_REPLICATE.

The source uses a circular layout, so the horizontal halo is shared between neighboring tiles rather than re-fetched for each one. Tiles then start at varying offsets inside the ring and their rows can straddle its end, so the primitive is handed the ring base and length and wraps both of the agens that read the source.

For images smaller than one tile the source ROI is expanded to a full tile (the out-of-image region is supplied by the same halo padding), while the destination tile is clamped to the image with a full-size tile arena, so the VMEM layout the kernel writes into never changes. This mirrors the approach used by the BilateralFilter operator.

Buffer Allocation#

The bank assignment is dictated by the primitive, which reads the input through agen port A, the scratch through port C and writes the output through port B:

Bank

Buffer

Layout

A

source tile

RDF_CIRCULAR with an nmsSize / 2 halo

B

destination tile

RDF_DOUBLE

C

horizontal-pass scratch, scalars, dataflow handlers

plain VMEM

Using three separate banks lets the compute read the source from A and write the scratch to C during the first pass, then read C and write B during the second, while the DMA streams the next source tile into A and the previous output tile out of B.

The tile is 64x64 pixels. The tile height is a multiple of 16 because the primitive over-runs its inner loops to the unroll factor and writes the output in blocks of 16 rows.

Kernel Implementation#

The NMS operator uses the NMS primitive to perform the suppression. Refer to the NMS primitive documentation for details, including:

  • The separable two-pass maximum filter that costs \(2N\) taps per pixel instead of \(N^2\)

  • The rank bias that implements the bottom-right tie-break without extra compare or select instructions, and the accuracy limitation that follows from it

  • Hardware instruction usage (vmax3 for the window maximum, dvmux for the branchless output selection)

One device binary is built per window size (pva_nms_K3_S32_dev and pva_nms_K5_S32_dev), so the primitive is instantiated at compile time and the maximum tree is fully unrolled.

Accuracy#

The window maximum is computed with a rank bias added to each tap. Because the bias is added to the pixel value, the decision is exact only where the center pixel differs from every other pixel of its window by more than \(B = (N^2 - 1) / 2\), i.e. 4 for a 3x3 window and 12 for a 5x5 window. Within that margin the result can err in either direction: a neighbor up to \(B - 1\) below the center pixel can still suppress it, and a neighbor up to \(B\) above the center pixel can fail to suppress it.

Performance#

Execution Time is the average time required to execute the operator on a single VPU core. Note that each PVA contains two VPU cores, which can operate in parallel to process two streams simultaneously, or reduce execution time by approximately half by splitting the workload between the two cores.

Total Power represents the average total power consumed by the module when the operator is executed concurrently on both VPU cores. Idle power is approximately 7.6 W on Jetson AGX Orin and 11.8 W on Jetson AGX Thor when the PVA is not processing data.

For detailed information on interpreting the performance table below and understanding the benchmarking setup, see Performance Benchmark.

Image Size

NMS Size

Execution Time

Submit Latency

Total Power

1920x1080

3x3

1.190ms

0.005ms

12.714W

1920x1080

5x5

1.496ms

0.006ms

12.412W

Image Size

NMS Size

Execution Time

Submit Latency

Total Power

1920x1080

3x3

1.339ms

0.003ms

15.36W

1920x1080

5x5

1.683ms

0.003ms

15.27W

Compatibility#

Requires PVA SDK 2.6.0 and later.