Channel Benchmarks

Holoscan channels are built on HoloMQ, a typed shared-memory transport. This page explains how to read their hot-path measurements for the two payload families you will use most: small POD values carried inline through the transport, and Tensors carried as descriptors while the payload bytes stay in place. Read the performance model first if you have not — the vocabulary here (serialized vs. saturated, delivery cycle vs. one-way E2E) comes from that page.

The controlled result set below comes from one Release build and one tuned x86_64 workstation. Later sections break the same run’s saturated rows into per-call costs to show the cost structure of each transport path. All values are measurements, not guarantees for your hardware.

Controlled transport suite

Intraprocess 1:1 latency

The HoloMQ concurrent-polling control reports a Google Benchmark mean rather than sample percentiles. It is included because its two-thread publication and receive loop is the closest available HoloMQ control, but its central statistic is not interchangeable with a Channel p50.

Interprocess 1:1 latency

Intraprocess 1:1 throughput

The HoloMQ control and Channel POD row both use concurrent polling. The Channel bulk rows use the same BufferHandle transfer mode as their interprocess counterparts, but publisher and subscriber run as threads in one process.

Interprocess 1:1 throughput

At fanout 1, logical messages/s and aggregate deliveries/s are identical.

For the directly matched CPU rows, Channel retains 90% of raw HoloMQ POD throughput and 92% of raw HoloMQ pinned-buffer throughput. The typed facade adds 0.852 µs to the POD p50 and 1.801 µs to the pinned-host Tensor p50. ImageT carries image metadata in addition to the Tensor descriptor, so it is not a raw-buffer equivalent.

The raw HoloMQ device row is the transport-only control for Channel’s descriptor-only rows: all device contents are initialized and synchronized before timing, so no per-message CUDA readiness or release fence is needed. Channel device Tensor throughput is 89% of that raw control. A separate HoloMQ synchronization sweep, shown below, measures the reusable-buffer protocol needed around per-sample GPU work. HoloMQ has no semantic Tensor, ImageT, or generated-schema type; its raw rows are controls, not substitutes for those types.

CUDA stream-ordering control

The device-buffer control separates raw descriptor transport from stream-ordered ownership. All four rows reuse backing initialized before timing; they differ only in the CUDA readiness and release protocol applied to each publication.

The fully stream-ordered row is the relevant lower-level control when a producer changes device contents on one stream and a consumer uses them on another. It enqueues a generation write, a generation wait, and deferred producer and consumer refcount releases. Its 66.989 µs pool-acquisition p50 shows that stream-ordered slot recycling, not the BufferHandle descriptor itself, limits this implementation under saturation. By contrast, the descriptor-only row needs no fence because initialization completed before publication.

HoloMQ serialized intraprocess cycles

These are Google Benchmark repetition-level mean iterations with one sample in flight, not per-sample percentiles. They are useful size controls but are not directly comparable with the concurrent latency table.

Controlled-run provenance

The controlled suite supplies the primary tables and the fully stream-ordered HoloMQ device latency. A focused rerun at the same SDK revision, on the same machine, supplies the four device synchronization throughput controls and the no-fence device latency.

The following command runs the same preset from the SDK source root on native x86_64 hardware. launch starts in the target’s build directory:

./run launch --arch x86_64 \
  --run-cmd "python3 ../benchmark/run_transport_benchmark_suite.py \
    --build-dir . \
    --output-dir transport-suite-channel-page-$(date +%Y%m%d-%H%M%S) \
    --preset balanced \
    --groups holomq_intraprocess,holomq_interprocess,channel \
    --cpus 2,3,4,5,6,7,8,9"

For the focused HoloMQ device control, the runner swept --device-sync none, publisher_ready, subscriber_release, and ready with queue depth 64 and pool size 66. The no-fence latency row used queue depth 1, pool size 3, and 200 µs pacing. The fully stream-ordered latency row came from the controlled suite’s ready case with queue depth 1 and 500 µs pacing.

The tables remain the accessible source of truth. No chart is included because the small number of curated rows is clearer in tabular form.

How to read the per-call tables

Two execution models appear on this page, and they measure different things:

• Serialized: one thread calls emit_sample() and then immediately calls receive_sample() on the paired endpoint, with exactly one sample in flight. The delivery cycle is the emit-plus-receive round through the transport. This isolates API cost cleanly.
• Saturated: publisher and subscriber run concurrently on separate threads or processes with no pacing and a queue depth of 64. The one-way E2E p50 runs from the publisher’s timestamp to completed receive and includes queue residence and scheduling. Because several samples are in flight, completed samples per second can exceed 1 / E2E — throughput is a pipelined completion cadence, not the reciprocal of latency.

Never compare serialized and saturated results directly; compare cost structure across them, not absolute values.

Every emit/receive in these benchmarks carries a full Sample<T> — the value plus its SampleMetadata — even when the convenience emit(T) / receive() overloads are used.

The tables below break the controlled run’s saturated rows into per-call costs. They use queue depth 64, no pacing, and 500 warmup samples, followed by 50,000 measured POD samples or 5,000 measured device Tensor samples. Each value is the median of three repetition-level results, and every row has the provenance listed under Controlled-run provenance.

POD inline transport

POD payloads travel inline: the bytes of T and the SampleMetadata are copied into the transport sample itself. The same code path serves same-process and interprocess deployments — only the queue’s process placement changes.

The near-identical intraprocess and interprocess rows are expected, not a fluke: both use the same typed inline transport path, so crossing a process boundary costs essentially nothing extra for POD payloads on this host. These queue-depth-64 rows complete slightly more samples per second than the queue-depth-1 POD rows in the throughput tables above.

Tensor transport: sidecar vs. BufferHandle

Tensor payloads never copy their bytes through the queue. Two mechanisms carry them:

• Same-process sidecar: the publisher stores the Tensor handle in a runtime-owned sidecar store and publishes a small descriptor. The subscriber resolves the descriptor and receives the same underlying Tensor allocation — no import, no new view.
• Interprocess BufferHandle: the publisher exports a descriptor encoding a BufferHandle plus shape, strides, and dtype. The subscriber imports the referenced pool slot (cached after first import) and materializes a new read-only Tensor view whose lifetime is tied to the lease.

Saturated comparison (1 MiB device Tensor)

Structurally, sidecar receive reuses the existing Tensor handle, whereas BufferHandle receive imports a pool slot and materializes a new Tensor view — that is why the interprocess receive p50 is about 400 ns higher even though its emit is the same. Both rows complete about the same number of samples per second, because publisher and subscriber pipeline across the thread or process boundary.

Putting the saturated 1:1 rows from both payload families side by side makes the cost structure visible at a glance: both POD deployments land at ~1.7 µs, and the two Tensor descriptor paths sit roughly 1.3–2.0 µs above them.

Fanout scaling

The same saturated rows at 1:1 and 1:2 fanout, with a 1 MiB device Tensor. Each publisher call is one logical sample regardless of fanout; the receive column is the worst subscriber’s successful receive_sample() p50 and the E2E column is the worst-subscriber one-way E2E p50.

Publisher cost grows with fanout in both modes, by 0.5–0.6 µs from 1:1 to 1:2, and becomes the limiting stage. Sidecar receive stays cheaper at both fanouts. The two transports complete about the same number of logical samples per second; their 1–2% difference is within the run-to-run spread.

Interpretation constraints

Read these before quoting any number:

• Device rows measure descriptor transport only. They transport descriptors, import pool slots, and materialize Tensor views; they do not copy 1 MiB of payload data or execute any real GPU work. CUDA generation-ordering waits and stream-ordered deallocation can dominate a real stream-ordered interprocess pipeline and are deliberately excluded here.
• Percentiles are post-warmup and successful-calls-only. p50 values are nearest-rank percentiles over individual successful samples after warmup; subscriber kNotReady polls are not included in the successful-receive durations.
• Saturated E2E is not idle-system latency. The saturated rows are throughput tests; their E2E percentiles include queue residence and scheduling. Use the paced latency tables above for one-way latency.

Recap

• Intraprocess Channel measured 0.964 million BenchmarkPod64 messages/s with 1.701 µs p50 latency; the closest concurrent HoloMQ polling control measured 1.124 million messages/s and a 1.409 µs mean iteration.
• In the controlled interprocess 1:1 run, HoloMQ delivered 64-byte values at 1.095 million messages/s with 0.804 µs p50 latency; Channel delivered BenchmarkPod64 at 0.982 million messages/s with 1.656 µs p50 latency.
• A 1 MiB device Tensor reached 549,331 messages/s and 3.133 µs p50 through Channel; device ImageT reached 407,820 messages/s and 4.231 µs p50.
• Tensor bytes never cross the queue: same-process sidecar reuses the original Tensor allocation, interprocess BufferHandle imports and materializes a new view.
• Serialized and saturated benchmarks measure different execution models — compare structure, never absolute medians.
• Publisher cost grows with fanout and becomes the limiting stage in both Tensor transports.
• Sidecar receive is about 400 ns cheaper than BufferHandle receive, but both Tensor paths complete about the same number of samples per second.
• Device rows exclude GPU work; treat them as transport cost, not pipeline latency.

Next: Core runtime benchmarks covers what the scheduler and operator activation machinery add on top of these channel costs. For the mechanics behind the interprocess path, see Interprocess channels and Standalone channel clients.