Core Runtime Benchmarks

The SDK ships its runtime performance coverage in-tree under the top-level benchmark/ directory, alongside its correctness tests. This page separates public transport-through-Graph measurements from implementation microbenchmarks used by SDK developers. Channel-focused transport results are covered on Channel Benchmarks. Application sensor-to-actuation measurements are not yet available and remain unpublished.

Transport through a running Graph

The controlled suite reports comparable payload and process-topology dimensions where matched HoloMQ, Channel, and Graph rows exist. Coverage is not identical across all three layers. These are independent measurement boundaries rather than additive substages: do not subtract one row from another to infer framework overhead.

Single-process Graph transport latency

Interprocess Graph transport latency

The table’s latency boundary is publication_to_receive: its source timestamp is taken immediately before emit(), after any producer-side Tensor loan acquisition and writes, and its sink timestamp is taken immediately after receive(). The docs preset uses descriptor-only raw Tensor and multi-Tensor rows. Their compiler-owned pool backing is initialized during warmup, so no backing memset occurs during measured publications; the rows isolate descriptor publication, Runtime execution, retention, and receive. The full callback and Tensor phase timings remain available in the artifacts, and the runner retains a full_write profile for separate producer-work diagnostics.

runtime_benchmark::MultiTensorMessageT contains one uint64 sequence plus two independent rank-1 uint8 pageable-host Tensor fields, first and second. Each field is 1 MiB and has its own compiler-owned pool, for a 2 MiB application payload. The interprocess 1:2 row uses queue depth 2 and 5 ms pacing to provide the flow-control headroom required by that topology.

The p99 and p99.9 values are descriptive. Seven repetitions characterize this controlled run but do not establish a tail-latency acceptance bound. The interprocess rows use at least 1,000 measured samples, which support p99 but not a consistently useful p99.9 estimate, so the interprocess table omits that column.

Single-process Graph throughput

Interprocess Graph throughput

The tables preserve logical publications/s and aggregate deliveries/s as separate metrics. For fanout, one logical publication produces one delivery per subscriber, so aggregate delivery rate scales with subscriber count. The small 1:1 difference between the rates is measurement-window alignment, not data loss. The paired single-process fanout rows compare one shared operator partition with separate source and subscriber partitions. Both arrangements measure the default process-local direct-retained fanout: the source’s emit() retains the sample for every subscriber directly, so neither needs an event wake or a channel-pump thread. The separate rows also give every subscriber its own partition on its own pinned CPU, which adds partition-owner wake and scheduling costs while allowing subscriber callbacks to run in parallel. The Notes column records the delivery path and wake mechanism present in each measurement.

For representative 1:1 rows, the running Graph measured 6.533–9.856 µs single-process and 16.324–28.950 µs interprocess p50 latency. This boundary includes Runtime wake-up, controller and any channel-pump coordination, readiness selection, operator activation, and receive materialization. It is therefore expected to cost more than the lower-level HoloMQ and Channel measurements, but the boundaries are independently timed and must not be subtracted to infer one internal stage.

Provenance and caveats

These measurements are evidence of cost structure on this machine, not performance guarantees for other hardware or application graphs.

Host checks confirmed that logical CPUs 2–9 were kernel-isolated (isolcpus, nohz_full, rcu_nocbs), their policies used the performance governor, and C1, C1E, and C6 were disabled. The suite artifact records the governor but does not yet capture CPU isolation or idle-state controls, so the generated provenance row remains conservative. The RTX A6000 also drove the workstation display, and GPU utilization was 6% immediately before the run, so GPU-sensitive rows may include that background load.

The two-process pinned-host Tensor and schema::ImageT rows come in pairs. The host-writer rows fill each buffer on the CPU, so the consumer has no readiness fence to wait for. The CUDA-stream-writer rows fill it on the producer’s stream, as a GPU producer would, so each slot returns to the pool only after the consumer’s stream passes that fence; the device rows share that handoff. Compare pinned-host rows only with the same writer. Both writers run below pageable-host schema::ImageT because a pinned-host or device consumer orders each slot’s release after its own CUDA stream, which costs CPU time on every sample. See Pinned-host readiness follows the writer for the contract and for when slot return limits throughput.

Reproducing the tables

Run from the SDK source root on a native Linux x86_64 benchmark host. Measurements on other hardware need their own provenance. First save the current CPU configuration and apply the benchmark controls:

sudo ./benchmark/configure_cpu_performance.sh \
  apply /tmp/holoscan-cpu-performance.state

Then run the rows used by this page in the SDK build container. launch starts in the selected target’s build directory, so the benchmark uses --build-dir .. The explicit x86_64 target matches the setarch command and published hardware:

QA_CPUS=$(./benchmark/configure_cpu_performance.sh recommend 8)
RESULTS_DIR="transport-suite-docs-core-runtime-$(date +%Y%m%d-%H%M%S)"
./run launch --arch x86_64 \
  --security-opt=seccomp=unconfined \
  --run-cmd "setarch x86_64 -R \
    python3 ../benchmark/run_transport_benchmark_suite.py \
    --build-dir . \
    --output-dir '${RESULTS_DIR}' \
    --preset docs-core-runtime \
    --cpus ${QA_CPUS}"

After the benchmark finishes, use the same host shell to generate the tables from the preserved artifacts and update this page in place. Query the directory for the same target instead of reconstructing its name:

BUILD_DIR=$(./run get_build_dir --arch x86_64)
python3 benchmark/summarize_runtime_benchmarks.py \
  "${BUILD_DIR}/${RESULTS_DIR}" \
  --update-doc docs/benchmarks_core_runtime.mdx

Finally, restore the saved host CPU configuration:

sudo ./benchmark/configure_cpu_performance.sh \
  restore /tmp/holoscan-cpu-performance.state

See Runtime benchmark hardware tuning for GPU clock controls, Jetson controls, verification, and recovery commands.

Internal suite inventory

Running the suites locally

All commands run from the repository root that contains the benchmark/ directory. The ./run bench helper executes inside the Holoscan build container with a Release build by default, rebuilds any stale benchmark targets, and runs the binaries directly so Google Benchmark output keeps its normal formatting:

./run bench --help    # full command reference and commented examples
./run bench --list    # list registered benchmarks (CTest name + short alias)
./run bench           # build what is missing and run all benchmarks

You can target one suite by alias or CTest name — for example ./run bench --name core_hello and ./run bench --name BENCHMARK_CORE_HELLO are equivalent. Suites are also registered with CTest under the benchmark label, so ctest -L benchmark runs them from an existing build tree.

Recap

• Controlled representative 1:1 Graph p50 latency ranged from 6.533–9.856 µs in one process and 16.324–28.950 µs between two processes.
• Single-process rows report p99.9 from at least 10,000 measurements; interprocess rows use at least 1,000 measurements and stop at p99.
• The generated tables compare shared and separate-partition 1:2 and 1:4 fanout, include a POD 10-operator single-process chain, and cover a generated schema with two preallocated Tensor fields.
• HoloMQ, Channel, and Graph measurements are independent boundaries; compare matched rows and do not subtract them.
• The internal microbenchmark inventory remains available for local diagnosis and CI smoke coverage; it is not the public transport result table.