YoloV5Decode#
Overview#
YoloV5Decode converts one YOLOv5 detection head into candidate detections: boxes, scores, class identifiers and a
count. YOLOv5 has three heads, one per scale, so a detector calls the operator three times. The three calls can append
to the same output tensors through a shared count, which produces one candidate list for non-maximum suppression without
a concatenation step.
YoloV5Decode is a custom operator for YOLOv5 detection heads, so it supports only the head format of the YOLOv5
models trained on COCO: exactly 80 classes, three anchors per grid cell and the YOLOv5 box formula. The anchor sizes
and strides (given as a tensor), the grid size, the confidence threshold, the image size for clipping, the head layout
(CHW or HWC) and the output capacity are parameters. A detector with another number of classes or another box
formula, such as an anchor-free YOLO model, requires a different decode.
In the YOLOv5 PVA+GPU pipeline, VPU1 runs the three decodes followed by BatchedNMS, and reads each head directly from the TensorRT output buffer.
Algorithm Description#
Each head holds, for each of three anchors at each grid cell \((g_x, g_y)\), the box offsets \(t_x, t_y, t_w, t_h\), an objectness logit and 80 class logits. With the anchor size \((a_w, a_h)\) and the head stride \(s\), the decoded box and the score of class \(c\) are:
Every class whose score is strictly greater than the confidence threshold produces one detection, so one anchor can produce several detections. The box corners are clipped to the network input. The results are in the order of the anchors, then of the classes.
Implementation Details#
Most anchors in a frame contain no object, and each remaining anchor requires the same computation for 80 classes. The implementation therefore rejects empty anchors with one scalar test, and processes the classes of the remaining anchors as vectors.
Dataflow Configuration#
The grid is split into tiles of 49 cells, that is 147 anchors. 49 is \(16 \cdot 3 + 1\), which provides the transposing CHW load a conflict-free VMEM line pitch.
A double-buffered RasterDataFlow brings the prediction tiles into VMEM. An HWC head arrives as
[49, 255]and is already anchor-major. A CHW head arrives as[255, 49].A double-buffered SequenceDataFlow brings the matching
[147, 5]anchor tile.A SequenceDataFlow reads the incoming count, which is the position where this call starts to append.
SequenceDataFlows write the boxes, scores, classes and the new count. The VPU sets their sizes to the number of new detections, so only valid data is written.
Buffer Allocation#
Superbank A holds the four F32 output box rows and the F16 score and S16 class compaction buffers.
Superbank B holds the anchor tiles, the CHW-to-anchor-major transpose buffer, the count, the parameters and the dataflow handlers.
Superbank C holds the double-buffered prediction tiles.
Kernel Implementation#
Layout conversion in the load. For a CHW head,
vfloat_load_transpconverts each tile to the anchor-major layout that an HWC head already has, so one decode kernel serves both layouts.Early rejection. A class score cannot exceed \(\sigma(obj)\). The host converts the threshold to a logit once, and the kernel skips every anchor whose objectness logit is not above it, without computing a sigmoid.
Classes as vectors. The 80 class logits of a surviving anchor are five full 16-lane vectors. The sigmoid uses the
fexp2andfrcpintrinsics.Compaction. The score comparison produces a lane mask, and
vcollate_idx_bitswithvpermutepacks the surviving classes. Because the compaction stores F16 scores, the kernel repeats the comparison in F32, so the rulescore > thresholdholds exactly at F16 rounding boundaries.Append. The count tensor is both input and output. Each call starts at the current count and adds its new detections, so three calls in stream order fill one candidate list. The caller sets the count to zero before the first call.
Limitations#
Exactly three anchors per cell and 80 classes.
The prediction and anchor tensors are tightly packed. The F32 box output can have row padding; the score, class and count outputs are packed.
At most 5,120 detections are stored. The incoming count is clamped to this capacity, and a call stops appending when the capacity is reached.
Performance#
Execution Time is the average time required to execute the operator on a single VPU core.
Note that each PVA contains two VPU cores, which can operate in parallel to process two streams simultaneously, or reduce execution time by approximately half by splitting the workload between the two cores.
Total Power represents the average total power consumed by the module when the operator is executed concurrently on both VPU cores.
Idle power is approximately 7.6 W on Jetson AGX Orin and 11.8 W on Jetson AGX Thor when the PVA is not processing data.
For detailed information on interpreting the performance table below and understanding the benchmarking setup, see Performance Benchmark.
Grid Size |
Input Layout |
Confidence Threshold |
Decoded Boxes |
Execution Time |
Submit Latency |
Total Power |
|---|---|---|---|---|---|---|
84x84 |
CHW |
0.25 |
1 |
1.147ms |
0.004ms |
10.213W |
84x84 |
CHW |
0.25 |
64 |
1.168ms |
0.004ms |
10.112W |
84x84 |
CHW |
0.25 |
512 |
1.323ms |
0.004ms |
10.112W |
84x84 |
CHW |
0.25 |
4096 |
2.552ms |
0.005ms |
9.81W |
84x84 |
CHW |
0.001 |
1 |
1.148ms |
0.004ms |
10.112W |
84x84 |
HWC |
0.25 |
1 |
0.714ms |
0.004ms |
10.213W |
42x42 |
CHW |
0.25 |
1 |
0.308ms |
0.004ms |
10.012W |
28x28 |
CHW |
0.25 |
1 |
0.152ms |
0.004ms |
9.911W |
21x21 |
CHW |
0.25 |
1 |
0.098ms |
0.003ms |
9.81W |
7x7 |
CHW |
0.25 |
1 |
0.037ms |
0.003ms |
9.103W |
Grid Size |
Input Layout |
Confidence Threshold |
Decoded Boxes |
Execution Time |
Submit Latency |
Total Power |
|---|---|---|---|---|---|---|
84x84 |
CHW |
0.25 |
1 |
1.289ms |
0.003ms |
13.752W |
84x84 |
CHW |
0.25 |
64 |
1.314ms |
0.003ms |
13.752W |
84x84 |
CHW |
0.25 |
512 |
1.488ms |
0.003ms |
13.752W |
84x84 |
CHW |
0.25 |
4096 |
2.880ms |
0.003ms |
13.752W |
84x84 |
CHW |
0.001 |
1 |
1.289ms |
0.003ms |
13.752W |
84x84 |
HWC |
0.25 |
1 |
0.802ms |
0.003ms |
13.854W |
42x42 |
CHW |
0.25 |
1 |
0.343ms |
0.003ms |
13.449W |
28x28 |
CHW |
0.25 |
1 |
0.169ms |
0.003ms |
13.854W |
21x21 |
CHW |
0.25 |
1 |
0.107ms |
0.003ms |
13.55W |
7x7 |
CHW |
0.25 |
1 |
0.038ms |
0.003ms |
13.045W |