Operator Performance Benchmarks#
Algorithm Performance Tables#
The PVA operator implementation pages feature comprehensive performance tables that offer in-depth insights into the performance of each operator on NVIDIA® Jetson AGX Orin™ and NVIDIA® Jetson AGX Thor™. Each table has a tab per platform; selecting a platform switches every table on the page. These tables include a variety of Key Performance Indicators (KPIs), such as mean execution time and mean submit latency. By examining these metrics, you can evaluate how the operator performs across different input sizes and configurations.
The KPI metrics are defined as follows:
- Execution Time
Average operator execution time on a single VPU core, including internal task scheduling overhead. Note that each PVA contains two VPU cores, which can operate in parallel to:
Process two streams simultaneously, or
Reduce execution time by approximately half by splitting the workload between the two cores.
- Submit Latency
The average time taken to submit the operator to the PVA. This submission operation is performed by the CPU.
- Total Power
The average total module power consumption during the operator’s execution. This measurement is taken when both VPU cores are simultaneously running identical workloads. Power consumption is measured using the tegrastats utility [1] and computed as the sum of the module power rails: VDD_GPU_SOC, VDD_CPU_CV and VIN_SYS_5V0 on Jetson AGX Orin, and VDD_GPU, VDD_CPU_SOC_MSS and VIN_SYS_5V0 on Jetson AGX Thor. Please note that, pre-regulator power is reported by tegrastats, which includes all regulator efficiency losses and module components power consumption. Idle power, which is measured when the PVA is not processing any data, is approximately as follows with the settings below:
7.6 W on Jetson AGX Orin
11.8 W on Jetson AGX Thor
Clock Frequency and Power Settings#
To ensure consistent measurements across runs, the following device frequency and power settings are applied before benchmarking.
NVIDIA® Jetson AGX Orin™ Settings:
Software: JetPack 7.2 (Jetson Linux R39.2.2), PVA SDK 2.10.0, operators built with the NVVPU device compiler
PVA/VPS freq.: 1.370 GHz
PVA/AXI freq.: 985.600 MHz
CPU: 12x Cortex-A78AE running at 2.202 GHz
EMC freq.: 3.199 GHz
Power mode: MAXN
Fan speed: MAX
NVIDIA® Jetson AGX Thor™ Settings:
Software: JetPack 7.2 (Jetson Linux R39.2.2), PVA SDK 2.10.0, operators built with the NVVPU device compiler
PVA/VPS freq.: 1.215 GHz
PVA/AXI freq.: 909.000 MHz
CPU: 14 Arm cores running at 2.601 GHz
EMC freq.: 4.266 GHz
Memory-controller and system-cache fabric freq.: 1.503 GHz
Power mode: MAXN
Fan speed: MAX