NMS (Non-Maximum Suppression)#
Overview#
The Non-Maximum Suppression (NMS) operator operates on a single-channel int32_t image, typically
a detector response map. For every pixel an \(N \times N\) window centered on that pixel is
searched; if the pixel is the maximum of the window it is copied to the output, otherwise the output
pixel is set to zero. Ties are resolved by position: among equal maxima within the window, the pixel
that appears last in raster order (left to right, top to bottom) wins, that is, the one with the
largest row index and, if those are equal, the largest column index. A rectangular plateau of equal
values therefore leaves exactly one surviving pixel, at its bottom-right corner, rather than keeping
all of them.
The current implementation supports 3x3 and 5x5 windows.
Nms Parameters: 5×5 window, constant border (value 0). The input is a Harris corner response; red dots are survivors above 5% of the maximum.
Algorithm Description#
Here \((u, v)\) ranges over the \(N \times N\) window centered on \((x, y)\), and earlier and later are in raster order within that window. The inequality against later taps is strict, which is what collapses a rectangular plateau of equal values to a single surviving pixel at its bottom-right corner instead of keeping every pixel of the plateau.
The output image has the same dimensions as the input. Pixels near the image border read their
out-of-image neighbors through the configured border mode, either NVCV_BORDER_CONSTANT with a
caller-supplied value or NVCV_BORDER_REPLICATE.
Implementation Details#
Dataflow Configuration#
Two RasterDataFlow streams move data between DRAM and VMEM: one reads the source with a halo of
nmsSize / 2 pixels in each direction, the other writes the destination. The halo is what supplies
the border pixels, so the kernel has no per-pixel boundary branches: PAD_CONST with the caller’s
border value for NVCV_BORDER_CONSTANT, PAD_BPE for NVCV_BORDER_REPLICATE.
The source uses a circular layout, so the horizontal halo is shared between neighboring tiles rather than re-fetched for each one. Tiles then start at varying offsets inside the ring and their rows can straddle its end, so the primitive is handed the ring base and length and wraps both of the agens that read the source.
For images smaller than one tile the source ROI is expanded to a full tile (the out-of-image region is supplied by the same halo padding), while the destination tile is clamped to the image with a full-size tile arena, so the VMEM layout the kernel writes into never changes. This mirrors the approach used by the BilateralFilter operator.
Buffer Allocation#
The bank assignment is dictated by the primitive, which reads the input through agen port A, the scratch through port C and writes the output through port B:
Bank |
Buffer |
Layout |
|---|---|---|
A |
source tile |
|
B |
destination tile |
|
C |
horizontal-pass scratch, scalars, dataflow handlers |
plain VMEM |
Using three separate banks lets the compute read the source from A and write the scratch to C during the first pass, then read C and write B during the second, while the DMA streams the next source tile into A and the previous output tile out of B.
The tile is 64x64 pixels. The tile height is a multiple of 16 because the primitive over-runs its inner loops to the unroll factor and writes the output in blocks of 16 rows.
Kernel Implementation#
The NMS operator uses the NMS primitive to perform the suppression. Refer to the NMS primitive documentation for details, including:
The separable two-pass maximum filter that costs \(2N\) taps per pixel instead of \(N^2\)
The rank bias that implements the bottom-right tie-break without extra compare or select instructions, and the accuracy limitation that follows from it
Hardware instruction usage (
vmax3for the window maximum,dvmuxfor the branchless output selection)
One device binary is built per window size (pva_nms_K3_S32_dev and pva_nms_K5_S32_dev), so the
primitive is instantiated at compile time and the maximum tree is fully unrolled.
Accuracy#
The window maximum is computed with a rank bias added to each tap. Because the bias is added to the pixel value, the decision is exact only where the center pixel differs from every other pixel of its window by more than \(B = (N^2 - 1) / 2\), i.e. 4 for a 3x3 window and 12 for a 5x5 window. Within that margin the result can err in either direction: a neighbor up to \(B - 1\) below the center pixel can still suppress it, and a neighbor up to \(B\) above the center pixel can fail to suppress it.
Performance#
Execution Time is the average time required to execute the operator on a single VPU core.
Note that each PVA contains two VPU cores, which can operate in parallel to process two streams simultaneously, or reduce execution time by approximately half by splitting the workload between the two cores.
Total Power represents the average total power consumed by the module when the operator is executed concurrently on both VPU cores.
Idle power is approximately 7.6 W on Jetson AGX Orin and 11.8 W on Jetson AGX Thor when the PVA is not processing data.
For detailed information on interpreting the performance table below and understanding the benchmarking setup, see Performance Benchmark.
Image Size |
NMS Size |
Execution Time |
Submit Latency |
Total Power |
|---|---|---|---|---|
1920x1080 |
3x3 |
1.190ms |
0.005ms |
12.714W |
1920x1080 |
5x5 |
1.496ms |
0.006ms |
12.412W |
Image Size |
NMS Size |
Execution Time |
Submit Latency |
Total Power |
|---|---|---|---|---|
1920x1080 |
3x3 |
1.339ms |
0.003ms |
15.36W |
1920x1080 |
5x5 |
1.683ms |
0.003ms |
15.27W |
Compatibility#
Requires PVA SDK 2.6.0 and later.