Operator Anatomy in Depth

You already know the shape of an operator from your first operator: three declarations — setup() for structure, contract() for time, compute() for work. This chapter goes deeper into the two declarations that surround the data itself: what setup() can actually say about each port, and the full menu that compute() has for reading inputs, publishing outputs, and talking to the runtime through its single ExecutionContext& argument.

Everything here follows one rule of the compose → compile → run model: an operator never owns or sizes a queue, allocates a transport buffer, or decides when it runs. It declares bounds and expectations in setup(), and at run time it reads a selection the runtime has already frozen for the activation. That division is what lets the compiler check your graph before a single sample flows.

Complete C++ example: examples/operator_anatomy/cpp

Reading inputs: one frozen selection

Before compute() runs, the runtime has already applied your temporal contract and frozen an input selection for this activation: which samples participate on each port. compute() only reads that frozen selection — it never pops a live queue. Input<T> offers owning reads, borrowed views, and inspectors:

The simplest path stays simple:

HOLOSCAN_TRY(auto value, in_.receive_data());   // just the value, copied out

selected_sample(): a callback-scoped scalar borrow

Use a borrow when the selected scalar value is only needed inside compute(). This is especially useful for descriptors such as ImageT: copying their generated strings, vectors, and nested tables can allocate even though no pixels are copied.

HOLOSCAN_TRY(auto view, in_.selected_sample());
const auto& value = view.data();       // const T&, no owning payload copy
const auto& metadata = view.metadata();
const auto& sample = view.sample();   // const Sample<T>&, the same frozen selection

*view and view-> also expose the selected const Sample<T>&. Repeated borrows refer to the same selection and do not consume the finite credits used by receive(). Missing, unbound, and batch-shaped inputs report errors through expected; this accessor neither waits for a sample nor converts a batch into a scalar.

Borrowing does not introduce a new queue-pressure policy. Owning reads and borrowed reads both leave Runtime’s selected input protected until callback retirement; releasing a local view is not a downstream-capacity acknowledgment. Depth-one queues can therefore reject the next publication during a short cross-partition overlap, even without sustained overload. See backpressure and queue sizing for lossless versus lossy handling and the limitations of retrying after a borrow or consuming Tensor-loan publication.

Python Operators use the same frozen scalar selection through a callback-scoped facade:

class Normalize(holoscan.Operator):
    inp = holoscan.Input("in", holoscan.payload.float32)
    def compute(self, context):
        selected = self.inp.selected_sample()
        metadata = selected.metadata
        # Decide whether this sample is usable before constructing its Python
        # payload value.
        if selected.is_stale or metadata.degraded:
            return
        value = selected.data
        # Use `metadata.frame_id` and `value` here.

SelectedSample itself is valid only on the callback thread and only until compute() returns. Its metadata property makes an owning SampleMetadata copy, and its data property lazily materializes an owning Python value through the input’s registered payload conversion. Those returned values may outlive the callback; retaining the SelectedSample facade and reading another property later raises NotReadyError. This Python path avoids an owning Core receive and does not take a read claim, but materializing a generated schema or configured Tensor view can still allocate or retain backing storage at the language boundary. Each access to data performs that projection, so assign it once if the value is used more than once.

The useful Python optimization is therefore conditional. Metadata-only telemetry, quality gates, stale-sample rejection, and multi-input arbitration can avoid payload materialization entirely. If an operator always reads data—especially for a small fixed scalar—the Python projection is still required and this API is not guaranteed to be faster than receive() or receive_data(). The allocation-free guarantee applies to the native C++ borrowed view, not to Python SelectedSample.data.

For unchanged ordinary generated schemas, forward the selection itself:

class ImuRelay(holoscan.Operator):
    inp = holoscan.Input("in", holoscan.schema.Imu)
    out = holoscan.Output("out", holoscan.schema.Imu)
    def compute(self, context):
        self.out.forward(self.inp.selected_sample())

This overload calls native Output<T>::emit_ref() without accessing .data, building a Python schema value, or copying the native object. Fields omitted from options inherit from the selected sample’s semantic metadata, just as with owning forward(); Runtime stamps publication sequence and time independently. The input and output must belong to this Operator, carry the same exact ordinary schema, and remain on the creating callback thread. Expired selections fail with NotReadyError. Tensor descriptors and other payload kinds are unsupported by this overload. Use owning data for edits or separate Tensor output loans for descriptors. Synchronous encoding and transport costs remain.

The benchmark/python_schema_io suite compares owning and selected metadata reads, data materialization, and forwarding with the SDK Imu schema. It reports callback-operation cost and separately traced Python transient-memory peaks; it does not count C++ allocations or promise an allocation-free Python callback.

GPU work submitted on ExecutionContext::cuda_stream() may finish after compute() returns: Runtime’s existing Tensor read leases and stream events protect the backing storage. That does not extend the host descriptor’s lifetime or authorize an unregistered side stream. For output metadata edits without owning descriptor copies, see transforming sensor payloads.

receive(): an owning Sample<T> with a read claim

receive() hands you a Sample<T> — a small owning wrapper with exactly two public members:

• sample.data — the payload value, of type T. This is exactly what receive_data() would have returned directly.
• sample.metadata — a plain-value SampleMetadata record of transport-neutral facts about the sample.

SampleMetadata carries the sample’s provenance. Author-supplied time-bearing fields are canonically absent (zero clock identity) unless a producer supplied them; publication time is the exception, because the runtime stamps it rather than the author:

HOLOSCAN_TRY(auto frame, camera.receive());                    // Sample<Tensor>
const holoscan::TimePoint captured = frame.metadata.capture_time();
if (frame.metadata.valid() && captured.valid()) {
  // compare `captured` against ctx.clock().now() for freshness, etc.
}

Interpreting TimePoint, capture versus publish time, and clock domains is covered in time and clocks.

The owning part matters: a Sample<T> holds a finite read claim on the underlying slot, which is what makes it safe to move the sample out of the callback and finish with it later. Claims come from a bounded pool sized at compile time — this is the same bounded-by-construction discipline as everywhere else in the runtime.

Batch views: selected_batch() and for_each_selected()

When the temporal contract selects a window of samples — the IMU input in the camera + IMU fusion pipeline is the canonical case — read them as a batch. selected_batch() returns a SelectedBatchView<T>: borrowed, immutable, and valid only for the duration of this activation. From the camera_imu_control example’s fusion operator:

if (imu.selection().state == holoscan::InputState::kSelected) {
  HOLOSCAN_TRY(auto batch, imu.selected_batch());
  for (const auto& sample : *batch) {
    integrated += sample.data;      // each element is a Sample<T> view
  }
}

for_each_selected(fn) is the same iteration with error propagation built in: the callable returns expected<void, Error>, and iteration stops at the first error, which becomes the call’s result.

Emitting outputs

Output<T> publishes a value or forwards a sample. Every overload is [[nodiscard]] expected<void, Error> — publishing can fail (an unbound output returns kNotReady), so every result must be checked. Dropping one is a compiler warning by design.

HOLOSCAN_TRY(out_.emit(value));                                           // by value
HOLOSCAN_TRY(out_.emit(value, holoscan::EmitOptions{.frame_id = 7}));     // with metadata
HOLOSCAN_TRY(out_.emit(std::move(sample)));                               // forward a Sample<T>

EmitOptions lets you attach author-owned metadata at publish time:

• .capture_time — the genuine acquisition instant, when you have one. It must belong to a clock admitted for the binding; omitting it is canonical absence, never a silent substitute timestamp. A present sample without this field remains valid arrival evidence, but capture-time selectors treat it as ineligible and apply the input’s authored missing-data policy.
• .frame_id — application frame/sample identity.
• .trace — an explicit trace context.
• .flags — validity/degradation flags.

The runtime stamps the framework-owned fields itself: sequence, payload kind, byte size, and publication time. Forwarding a received Sample<T> with emit(std::move(sample)) preserves its author metadata (capture time, frame_id, trace, flags) downstream — which is why that overload deliberately takes no EmitOptions: the sample already carries them.

For pooled values there is a loan path — allocate() returns a move-only LoanedSample<T> you fill and then emit(std::move(loan)) — and Output<Tensor> replaces generic allocate() with allocate_tensor(request) → a move-only TensorOutputLoan. Tensor production, host/device writers, and commit-based stream ordering are covered in tensors in depth; the pool-not-heap semantics of “allocate” are introduced in tensors 101.

max_emits_per_compute: declaring your burst

A port emits one sample per compute() by default. If an operator may publish more than once to the same port in a single activation, declare that bound in setup():

spec.output(out_, "out").max_emits_per_compute(4U);   // up to 4 emits per activation

This is not a soft hint. The bound sizes the output pool at compile time, so the runtime can preallocate exactly enough slots for your worst-case burst — and an emit beyond the declared bound fails rather than growing anything. The common case needs no call at all: declaring the port plainly means “one emit per compute.”

The setup() surface: spec builders

spec.input(member) and spec.output(member) (optionally with a name) return chainable builders. Everything you can say about a port is said here, once, before compilation — the compiler validates it against the rest of the graph.

InputSpecBuilder (from spec.input(...)):

• .queue_depth(n) — an explicit, exact logical input-history depth. The compiler preserves n; it does not replace it with a rate-derived recommendation or a provider’s physical ring size. When depth is omitted and exact nominal-rate plus consumption-contract evidence closes the sizing problem, the compiler derives the logical depth automatically. Only an otherwise unresolved legacy queue uses the depth-one fallback. The rate-aware ports example shows both an automatically derived depth and an explicit override.
• .expects_tensor(spec) — declares a TensorInputSpec containing the representation, bounds, and optional conversion this operator expects, letting the compiler reject mismatched producers instead of you discovering them at run time.
• .payload_options(...), .diagnostic_alias(...) — payload tuning and a stable alias for diagnostics.

OutputSpecBuilder (from spec.output(...)):

• .max_emits_per_compute(n) — the burst bound described above.
• .produces_tensor(spec) — the output-side spec, the mirror of expects_tensor.
• .diagnostic_alias(...) — as on inputs.

TensorOutputSpecBuilder — spec.output(...) on an Output<Tensor> returns this specialized builder. Runtime-pool storage intent inside the output spec is the only grant of direct Tensor allocation authority:

spec.output(frame, "frame")
    .produces_tensor(holoscan::TensorOutputSpec{
        .representation = kHostByteFrame,
        .bounds = holoscan::tensor_bounds(64U),
        .storage = holoscan::TensorOutputStorage::kRuntimePool,
    });

Readiness sources live on the spec too, at the OperatorSpec level rather than per port: spec.readable_source(...), spec.notification_source(...), spec.doorbell_source(...), and spec.spin_source(...) declare named, operator-owned event sources — an OS-readable handle, a cross-thread notification, a device-completion doorbell, or a polled condition — that your contract() can then name as triggers (for example OnReadable{...}). Declaring the source in setup() and referencing it in contract() keeps the structure/time split clean. Their use as triggers is covered in temporal contracts.

Lifecycle callbacks are the spec’s other operator-level surface: spec.lifecycle().stage(...)/.optional(...) registers sparse advanced stage callbacks, plus produces/requires_dependency ordering facts. Simple start()/stop() overrides need no spec declaration at all. Both paths are covered in lifecycle callbacks.

The ExecutionContext facade

The single ExecutionContext& argument to compute() is a one-pointer, activation-scoped facade. Every accessor is O(1), non-blocking, and allocation-free, and it deliberately exposes no graph mutation or scheduler wait — compute() cannot reach outside its activation.

• Parameters — ctx.params<Params>() returns the immutable parameter aggregate pinned for this callback; ctx.params_generation() identifies which generation you are seeing. See parameter schemas.

• Time — ctx.clock() (a read-only ClockRef), ctx.tag() (this activation’s identity), ctx.tag_time() (its deterministic logical coordinate), and ctx.activation_time() (dispatch time in the activation clock — distinct from any sample’s capture time). See time and clocks.

• Stream — ctx.cuda_stream() returns the CUDA stream assigned to this activation; received tensors are already ordered on it, so you launch work and never synchronize on the hot path.

• Tracing and status — ctx.trace() / ctx.trace_id() for correlation, ctx.activation() for the activation and violation outcome.

• Identity: ctx.operator_path() returns the canonical percent-encoded authored path of this Operator, such as a/b%2Fc, for callback-time logs and diagnostics. The borrowed view must not be retained after compute().

• Cancellation — ctx.cancellation().stop_requested() observes a cooperative stop. A long-running or looping compute() should poll it and wind down cleanly:

  if (ctx.cancellation().stop_requested()) {
    return {};   // wind down cooperatively
  }

  How stops propagate through a running application is covered in running and stopping.

• Whole-run termination requests — ctx.request_completion() requests immediate whole-run success (kRequestedCompleted), while ctx.request_stop() requests kStopped. Both close session-wide callback admission: claims ordered before that boundary may proceed and retire, but no claim ordered afterward enters. Stop overrides completion, and callback or cleanup failure overrides both, with one exception: a callback that fails only because the stop cut one of its publications short still ends the run stopped, with a warning (see publications blocked at stop). Requested completion is not EOS and does not guarantee source closure, downstream drain, or queued delivery.

• Restart — ctx.request_restart(options) (plus an authorized-target overload) returns expected<LifecycleRestartReceipt, Error>. Acceptance only queues the request; the lifecycle machinery executes it. The full restart model lives in restart control.

What can cross a port: PublishedPayload

Not every C++ type may travel between operators. A port admits only PublishedPayload types:

• Arithmetic types and enums — plain scalar values. Enums are limited to same-process typed Graph edges; use their fixed-width underlying type across process or external boundaries.
• Exactly Tensor — the SDK’s pooled, stream-ordered payload (not tensor-like wrappers).
• Schema-identified copyable values — structs you have explicitly registered with a payload schema.

Emitting anything else is a compile-time error, not a runtime surprise. This restriction is deliberate: every admitted type has known size, copy, and serialization behavior, which is what lets the compiler size pools and — for schema-identified types — carry the same payload across process boundaries. The full admission rules, and how to make your own struct schema-identified, are in payload type rules.

Recap

• selected_sample() borrows scalar data and metadata without taking a claim. Keep the C++ view or Python facade within the callback; Python .data lazily creates an owning language projection.
• receive() returns an owning Sample<T> (payload in .data, provenance in .metadata) and takes a finite read claim; receive_data() copies just the value; selected_batch() / for_each_selected() read windowed inputs through activation-scoped views. None of them pop a queue — the selection is frozen per activation.
• Every emit() is [[nodiscard]] expected; EmitOptions attaches author-owned metadata, and forwarding a Sample<T> preserves it.
• setup() builders declare everything the compiler needs: queue_depth, direction-specific Tensor specs and runtime-pool intent, max_emits_per_compute (which sizes the output pool), and readiness sources for contract() to trigger on.
• ExecutionContext is the callback-lifetime runtime surface inside compute(): Operator path, params, clock, tag, CUDA stream, cancellation, whole-run termination requests, and restart requests. Its queries are O(1); termination requests are bounded, non-blocking, and allocation-free.
• Only PublishedPayload types cross ports: arithmetic/enums, exact Tensor, and schema-identified copyable values.

Next: temporal contracts — how contract() tells the runtime when to invoke the compute() you just learned to write.