Reference

GPU Telemetry & Power-Efficiency Metrics Dataflow

View as Markdown

How per-vendor GPU power/energy signals flow from the collectors to the console. NVIDIA and AMD stay in separate lanes end-to-end: they converge only in the shared TelemetryHierarchy store, then re-split at the per-platform aggregation so each vendor’s totals sum only its own GPUs. A mixed NVIDIA+AMD run emits both lanes’ sections; a vendor with no reporting GPU is omitted entirely.

A self-contained, richly-styled HTML version of this diagram lives alongside this page at gpu-telemetry-metrics-dataflow.html (open it directly in a browser).

Dataflow

Lanes

  • NVIDIA lane (green): DCGMTelemetryCollector and PyNVMLTelemetryCollector write nvidia_* fields on TelemetryMetrics. The DCGM path maps raw DCGM field names via DCGM_TO_FIELD_MAPPING; both stamp platform="nvidia".
  • AMD lane (red): AMDSMITelemetryCollector writes amd_power / amd_energy_consumption and stamps platform="amd".
  • Shared (blue): the single TelemetryHierarchy, the aggregation entry point, the token/concurrency inputs, the console_group routing, the warning banner, and the per-GPU / file exporters.

Convergence and re-split

The two lanes meet in exactly one place: GPUTelemetryAccumulator stores every record — regardless of vendor — in a single TelemetryHierarchy, with each GPU tagged by metadata.platform. compute_efficiency_metrics then iterates _EFFICIENCY_VENDORS and calls _sum_gpu_power_watts / _sum_gpu_energy_joules with a (platform, field) pair, so each vendor’s sums include only its own GPUs. This is what keeps a mixed cluster’s NVIDIA and AMD totals from blending.

Resulting console order

The disclaimer banner renders first, so it’s clear everything below it is vendor-specific:

╭─ GPU Telemetry Platform ─╮  Platform: nvidia, amd · metric semantics are platform-specific
GPU Power Efficiency (NVIDIA)   Total GPU Power · Total GPU Energy · Output Tokens per Joule · Energy per User  (avg only)
GPU Power Efficiency (AMD)      Total GPU Power · Total GPU Energy · Output Tokens per Joule · Energy per User  (avg only)
AIPerf | GPU Telemetry Summary  per-GPU tables (power, util, mem, temp, …)

Source of truth

  • src/aiperf/gpu_telemetry/ — collectors, accumulator.py, constants.py
  • src/aiperf/metrics/types/power_efficiency_metrics.py — the eight vendor metric classes
  • src/aiperf/exporters/console_*_exporter.py — the disclaimer, per-vendor efficiency, and telemetry exporters