Advanced Temporal Patterns

Most operators get by with the temporal vocabulary from Temporal contracts: one trigger (OnEach, OnClock, OnAligned, OnWindow), one selector per input (Exact, LatestWithin, DrainAll), and a missing-data policy. This page covers the advanced tier for pipelines where that vocabulary is not enough: predicate-driven activation with OnPredicate, all-or-nothing batch selection with BracketedWindow, and composition recipes for multi-rate graphs where different inputs arrive at very different cadences.

Complete C++ examples: examples/temporal_contracts/on_predicate/cpp and examples/temporal_contracts/on_window/cpp

When the basics run out

The core trigger vocabulary answers “activate on a sample”, “activate on a schedule”, or “activate when heads align”. Two situations fall outside it:

• Data-dependent activation. You want to run compute() only when the content of the inputs satisfies a condition — a threshold crossing, a state-machine edge, a match between two streams — not merely when a sample arrives.
• Neighborhood selection. One activation needs a bracket of samples from a fast input over the interval between two slow anchors, plus readings just outside that interval — rather than one freshest sample or a queue drain.

OnPredicate addresses the first; BracketedWindow the second. Both lower to explicit records in the compiled plan, so they stay inspectable with the same plan tooling as everything else, and both keep the runtime’s bounded-by-construction guarantees.

OnPredicate: data-dependent activation

OnPredicate evaluates a typed callable — the predicate — and creates a compute candidate when it returns true. The trigger has three parts:

holoscan::Contract contract() const override {
  auto evaluator = holoscan::predicate(
      "pressure-threshold",
      [](const holoscan::PredicateEvaluationContext&,
         const holoscan::PredicateInputView<Reading>& driver_view) noexcept {
        return driver_view.sample().data.pressure > 90.0F;
      });
  holoscan::Contract c;
  c.trigger(holoscan::OnPredicate{
       .evaluate_on = holoscan::OnInputUpdate{sensor},
       .predicate = std::move(evaluator),
       .firing = holoscan::PredicateFiring::kOnFalseToTrue,
   })
   .input(sensor, holoscan::Exact{});
  return c;
}

• evaluate_on names when the predicate runs. There are exactly two sources: OnInputUpdate{port} evaluates once for every accepted sample on a driver input, and OnClock{.period = ...} evaluates on a periodic schedule. The driver’s sample is pinned as the first typed view the evaluator sees.
• predicate is a binding built with holoscan::predicate(...). The factory copies your callable into a cold descriptor — the contract owns its own deep copy, so no authoring address ever enters a plan or a run session.
• firing decides when a true result becomes an activation. PredicateFiring::kOnFalseToTrue (the default) fires only on the edge — the remembered result must change from false to true. Use PredicateFiring::kOnEveryTrueEvaluation when every satisfied evaluation should activate.

To evaluate over more than the driver, declare additional scalar inputs with holoscan::predicate_inputs(...); the evaluator then receives one frozen PredicateInputView<T> per declared input, in declaration order, after the driver view:

auto evaluator = holoscan::predicate(
    "match-condition",
    holoscan::predicate_inputs(reference),
    [](const holoscan::PredicateEvaluationContext&,
       const holoscan::PredicateInputView<Reading>& driver_view,
       const holoscan::PredicateInputView<Reading>& reference_view) noexcept {
      return driver_view.sample().data.id == reference_view.sample().data.id;
    });

Selector and missing-policy semantics for predicate inputs are still declared through the ordinary .input(...) / .on_missing(...) calls on the contract; the predicate only reads what the contract selected.

Bounded, nonthrowing callables

A predicate runs inside the scheduler’s hot decision path, so the API is designed to keep evaluation cheap and safe:

• Return bool (or expected<bool, Error>), take noexcept seriously. Evaluators return true/false; a fallible evaluator returns holoscan::expected<bool, holoscan::Error> and its errors follow the ordinary failure path — there is no throwing escape hatch from evaluation.
• State is transactional. If your predicate carries mutable state (for edge detection across evaluations, say), declare it as a type parameter. The runtime evaluates against a candidate copy and commits or discards it atomically, so a failed evaluation never corrupts the committed state. The state type must be default-initializable with nothrow copy, swap, and destruction — the compiler enforces this as a concept, so an unsuitable type fails at the predicate(...) call site.
• Callable form is classified for admission. The factory records what kind of callable you supplied — a free function, a noncapturing lambda, a fixed-size inspectable functor, or an opaque capturing closure. Constrained deployment profiles admit only the forms whose cost and behavior they can bound; an opaque closure that is fine in a development profile may be rejected when the plan is compiled for a constrained one. Prefer noncapturing lambdas or small trivially copyable functors, and move any tunable values into the explicit config argument of predicate(...) — configuration is copied, immutable, and identity-hashed for the plan.
• Identity is part of the contract. The first argument to predicate(...) is a stable name. For predicates whose decisions must be attributable across runs and replays, use a qualified holoscan::PredicateKey{.qualified_name = "...", .semantic_version = {1, 0, 0}} instead of a bare string; the version is recorded as replay evidence. See Determinism and replay for how that evidence is used.

BracketedWindow: all-or-nothing neighborhoods

BracketedWindow selects a bounded, all-or-nothing batch around the closed capture-time interval between consecutive anchors. It is paired with the OnWindow trigger. The first camera anchor establishes the starting boundary. The second opens the first interval. This describes the interval between frames, not the exposure duration of either frame.

holoscan::Contract contract() const override {
  using namespace std::chrono_literals;
  holoscan::Contract result;
  result
      .trigger(holoscan::OnWindow{.anchor = camera, .max_wait = 20ms, .max_pending_windows = 2U})
      .input(camera, holoscan::Exact{})
      .input(imu, holoscan::BracketedWindow{.lookbehind = 1U, .lookahead = 1U, .max_samples = 16U})
      .on_missing(imu, holoscan::SafeDefault{});
  return result;
}

This is the contract shape in the camera + IMU walkthrough. The anchor must use Exact. The IMU batch includes:

A complete bracket, then an overlapping bracket

The numbers below are capture timestamps in illustrative units, not sample indices. Every IMU sample in this trace is shown, with no intermediate samples omitted. Use lookbehind = lookahead = 1 and max_samples = 8. Anchor 10 primes, then anchor 20 creates [10, 20]. IMU 20 is an interior sample, so it cannot supply the required successor. The bracket closes when IMU 25 arrives:

• Green-outlined circles: selected samples inside the interval, including its endpoints.
• Green-outlined diamonds: selected strict predecessors or successors outside the interval.

Both shapes belong to the batch. The sample captured at 15 changes from a circle in W1 to a diamond in W2 because it becomes a strict predecessor. max_samples counts all selected samples, including the outer diamonds. W1 selects five samples and W2 selects four, so the bound of eight constrains neither.

After anchor 30 arrives, IMU 35 completes W2. No IMU sample at 30 is required. This is an engine-level membership trace, with no clock or deadline claim.

Each selection is a non-destructive snapshot of retained samples. The two batches share the samples captured at 15, 20 and 25. The sample at 20 is interior to both intervals. Application integration must account for interval endpoints and shared support samples rather than blindly accumulating every value in every batch. DrainAll instead consumes a finite FIFO survivor cut. Accepted drains do not overlap and do not use this capture-time interval.

Pending, selected, or terminally missing

A pending window has not yet failed. It holds no worker in a waiting callback and does not invoke the missing policy on every readiness pass.

Each step uses the same capture-time positions, with lookbehind = 1, lookahead = 1 and max_samples = 8. Step captions give arrival order. Horizontal distance is capture time, not processing time. The window stays pending until a sample strictly after 20 exists. The engine then checks the successor, predecessor history and total size, in that order. The five selected samples are within the size limit.

Each result appears beside its case. A complete bracket is selected. A terminal reason leaves no selected batch, and the declared missing policy decides what follows. SafeDefault permits a degraded activation with the reason and no IMU samples, while Skip produces no activation. Without a sample before 10, the engine still waits for 25 before reporting kInsufficientHistory. With max_samples = 4, the complete five-sample bracket reports kWindowSampleLimit instead of delivering a partial batch. That bound is valid because it is at least lookbehind + lookahead + 1 = 3. Any activation still depends on the rest of the contract.

A closed result can become an activation subject to the rest of the contract. Exact selects the stored current camera anchor, 20 here. A close witness supplies the activation’s logical close coordinate.

Declare Skip{} or SafeDefault{} explicitly for the bracketed input. ReuseLast{} is invalid for batch selectors and is rejected during contract validation. Both lookaround counts must be positive. max_samples counts the outer samples and the entire inclusive interior. It must admit at least lookbehind + lookahead + 1. Use a blocking queue: drop-oldest policy is rejected. An explicit queue depth must cover max_samples * max_pending_windows. An omitted depth is derived from that product. Size both bounds for the expected rates and burstiness: a 1 kHz IMU with a 60 Hz camera can require about 17 interval samples plus brackets, so a bound of 16 is too small.

A pending-limit closure still advances the previous-anchor boundary to the new anchor. The following interval starts there, leaving the skipped interval separate.

Silent is different from ended. While the IMU stream remains open but silent, windows can time out and SafeDefault can produce degraded camera-only activations. When the IMU ends, incomplete pending windows close with kEndOfStream. Closing a window does not itself guarantee an immediate activation. While the camera input remains open, the IMU end-of-stream event alone cannot trigger one.

Later camera anchors create no new intervals. A later arrival can still trigger an activation for a previously pending window, using its terminal result and retained camera anchor. SafeDefault permits a degraded activation if the rest of the contract admits it. Skip suppresses it. After outstanding results are handled, SafeDefault cannot sustain camera-only fusion from new camera frames. The session and camera can keep running.

Capture time, waiting, and compute time

Capture time decides membership. The current implementation also derives timeout eligibility from the oldest pending window’s current-anchor capture time, converted to its wait clock, plus max_wait. It does not restart the wait when the windowing logic ingests that anchor.

The following worked example uses one admitted clock and a shorter wait than the 20 ms contract above:

// In contract() above, with std::chrono_literals in scope:
result.trigger(holoscan::OnWindow{.anchor = camera, .max_wait = 10ms});

A camera frame captured at 10 ms has already primed the selector. The next frame is captured at 20 ms, and the windowing logic ingests it at 28 ms. No IMU sample after 20 ms has arrived, so the interval [10, 20] opens as a pending window. Its timeout threshold is 20 ms + 10 ms = 30 ms. At or after 30 ms, a readiness pass can find the window still pending and enqueue a timeout witness. When a later pass ingests that witness, the window closes with kWindowTimeout and the missing policy decides what follows. SafeDefault permits a degraded activation with no IMU samples. Skip suppresses the activation.

The figure follows the still-pending path. If ingestion is delayed beyond max_wait, the window is eligible for timeout immediately. If an IMU successor is ingested before the timeout witness is generated, the window closes through the usual successor, history and size checks instead. A complete bracket that is already cached closes as soon as the frame is ingested. The wait counts from capture time, so 28 ms + 10 ms = 38 ms is not the threshold. Eligibility, timeout observation and compute are separate events, and none has a promised time.

A producer restart, a capture timestamp that goes backwards, or a clock-domain change resets retained history and pending windows. A discontinuous anchor primes again. After a sample-side reset, the next anchor primes. No interval bridges the reset. Do not rely on undelivered closed results surviving a producer restart. Lifecycle readiness and restart are separate from data alignment. See Lifecycle and Restart in Depth.

Read the selected batch

Use selected_batch() or for_each_selected(...). receive() is scalar-only and reports a cardinality mismatch for a batch-selected input:

if (imu.selection().state == holoscan::InputState::kSelected) {
  auto batch = imu.selected_batch();
  if (!batch) {
    return holoscan::make_unexpected(std::move(batch).error());
  }
  // Application helper: use interval endpoints and avoid double-counting
  // shared support samples. Its implementation depends on the sensor model.
  process_window(*batch);
}

process_window above is an illustrative application helper, not an SDK API. The immutable view is valid only during the current activation, is sized and indexable, and can be iterated repeatedly. It is not contiguous storage: there is no data() pointer or span conversion. Retain derived application state, not the view or references into it.

Multi-rate composition recipes

The building blocks above compose into a small set of recurring shapes for graphs whose inputs run at different rates. Each recipe lowers to explicit plan records — read sets, sample bounds, timing — so compile() can check it and the tooling can show it.

Fast anchor, slow context

One input owns the cadence; the others contribute their freshest value. Trigger on the fast input, select the slow ones with LatestWithin, and decide staleness policy explicitly:

c.trigger(holoscan::OnEach{fast})
 .input(slow, holoscan::LatestWithin{.max_age = 100ms})
 .on_missing(slow, holoscan::ReuseLast{.max_age = 200ms});

This is the workhorse pattern — the slow/fast fusion example is built on it. OnEach pins the fast driver implicitly; only the side input needs an explicit selector. Use this pattern whenever one stream is clearly the beat-keeper.

Symmetric alignment

When no input owns the cadence and you need samples that belong together, use OnAligned over a closed port set with an inclusive head-spread window, selecting every member with Exact:

c.trigger(holoscan::OnAligned{.ports = holoscan::ports(camera, imu), .window = 2ms})
 .input(camera, holoscan::Exact{})
 .input(imu, holoscan::Exact{});

Alignment consumes deterministically from the FIFO heads; members not selected with Exact are rejected when the plan is compiled, not at the .trigger(...) call site.

Windowed integration

When each slow anchor needs the neighborhood of a fast stream rather than one aligned sample, combine OnWindow with BracketedWindow as shown in the previous section. The first anchor primes history. Subsequent anchors define intervals. Window closure can make an activation eligible, without blocking a callback. max_wait controls timeout eligibility as described above, while max_pending_windows bounds simultaneously pending intervals.

Clock-paced batch drain

For consumers that should run at their own rate and absorb everything that arrived since the last activation — statistics, logging, batch upload — pair a periodic trigger with a consume-all batch:

c.trigger(holoscan::OnClock{.period = 33ms})
 .input(events, holoscan::DrainAll{});

Late ticks coalesce (only the most recent due tick fires, preserving phase), and the batch bound is derived from the queue geometry at compile time. Read the result with selected_batch(); a valid empty batch is an empty view, not an error. For clock selection and time domains, see Time and clocks.

Derived and divided rates

To run an operator at a fraction of an input’s rate, prefer making the rate relationship a declared parameter of your operator and let the contract express it with the vocabulary above (for example, an OnClock period derived from a typed parameter) rather than counting samples imperatively inside compute(). A declared rate lands in the plan where tooling can see it; a hidden counter does not.

Recap

• OnPredicate activates on a data-dependent condition: pick an evaluation source (OnInputUpdate or OnClock), a firing rule (edge-triggered kOnFalseToTrue by default), and keep the callable cheap, nonthrowing, and admission-friendly — noncapturing or small and trivially copyable.
• Predicate state is transactional and constrained to nothrow operations; configuration is immutable and identity-hashed; qualified PredicateKey identities feed replay evidence.
• BracketedWindow delivers an all-or-nothing, non-destructive, event-time-ordered bracket around consecutive anchors. Size max_samples for the worst case, because overflow delivers nothing and reports kWindowSampleLimit.
• Batch-selected inputs are read with selected_batch() / for_each_selected(...), never receive().
• Four recipes cover most multi-rate graphs: fast anchor + LatestWithin, OnAligned + Exact, OnWindow + BracketedWindow, and OnClock + DrainAll.

Next: Interprocess channels.