Operator Performance Benchmarks#

Algorithm Performance Tables#

The PVA operator implementation pages feature comprehensive performance tables that offer in-depth insights into the performance of each operator on NVIDIA® Jetson AGX Orin™ and NVIDIA® Jetson AGX Thor™. Each table has a tab per platform; selecting a platform switches every table on the page. These tables include a variety of Key Performance Indicators (KPIs), such as mean execution time and mean submit latency. By examining these metrics, you can evaluate how the operator performs across different input sizes and configurations.

The KPI metrics are defined as follows:

Execution Time

Average operator execution time on a single VPU core, including internal task scheduling overhead. Note that each PVA contains two VPU cores, which can operate in parallel to:

  • Process two streams simultaneously, or

  • Reduce execution time by approximately half by splitting the workload between the two cores.

Submit Latency

The average time taken to submit the operator to the PVA. This submission operation is performed by the CPU.

Total Power

The average total module power consumption during the operator’s execution. This measurement is taken when both VPU cores are simultaneously running identical workloads. Power consumption is measured using the tegrastats utility [1] and computed as the sum of the module power rails: VDD_GPU_SOC, VDD_CPU_CV and VIN_SYS_5V0 on Jetson AGX Orin, and VDD_GPU, VDD_CPU_SOC_MSS and VIN_SYS_5V0 on Jetson AGX Thor. Please note that, pre-regulator power is reported by tegrastats, which includes all regulator efficiency losses and module components power consumption. Idle power, which is measured when the PVA is not processing any data, is approximately as follows with the settings below:

  • 7.6 W on Jetson AGX Orin

  • 11.8 W on Jetson AGX Thor

Clock Frequency and Power Settings#

To ensure consistent measurements across runs, the following device frequency and power settings are applied before benchmarking.

NVIDIA® Jetson AGX Orin™ Settings:

  • Software: JetPack 7.2 (Jetson Linux R39.2.2), PVA SDK 2.10.0, operators built with the NVVPU device compiler

  • PVA/VPS freq.: 1.370 GHz

  • PVA/AXI freq.: 985.600 MHz

  • CPU: 12x Cortex-A78AE running at 2.202 GHz

  • EMC freq.: 3.199 GHz

  • Power mode: MAXN

  • Fan speed: MAX

NVIDIA® Jetson AGX Thor™ Settings:

  • Software: JetPack 7.2 (Jetson Linux R39.2.2), PVA SDK 2.10.0, operators built with the NVVPU device compiler

  • PVA/VPS freq.: 1.215 GHz

  • PVA/AXI freq.: 909.000 MHz

  • CPU: 14 Arm cores running at 2.601 GHz

  • EMC freq.: 4.266 GHz

  • Memory-controller and system-cache fabric freq.: 1.503 GHz

  • Power mode: MAXN

  • Fan speed: MAX

Reference#

  1. https://docs.nvidia.com/drive/drive_os_5.1.6.1L/nvvib_docs/index.html#page/DRIVE_OS_Linux_SDK_Development_Guide/Utilities/util_tegrastats.html