Runtime Parameter Updates

Sometimes a parameter should change while the graph is running: nudging a gain, adjusting a threshold, retuning a filter interval — without stopping the session, recompiling, and starting over. Holoscan 5.0 supports this through runtime parameter generations: a running operator’s typed parameters live in immutable, versioned snapshots, and an update publishes a new snapshot at a scheduler-safe boundary rather than mutating the current one in place.

This page covers the whole cycle: how a field becomes updatable, how an operator reads the values pinned for its current callback, how application code submits and observes an update through a control handle, how operator code submits runtime_mutable() self-updates from inside compute(), and how to fence off further updates.

For a model-catalog example that separates validation, resource replacement and old-epoch restoration, see Lifecycle and Restart in Depth.

Complete C++ examples: examples/experimental/runtime_params/cpp and examples/experimental/web_tuner/cpp

Parameters as immutable generations

When a session starts, each operator is bound to immutable generation 0. This is the plan’s compile-resolved initial aggregate: the parameter type’s default aggregate, optionally replaced by an explicit typed baseline supplied through OperatorHandle::set_params(), and then sparsely patched by an admitted graph-owned parameter input. See Typed parameters and Configuring applications.

An update never mutates a published generation. Instead, an accepted update reserves a new immutable candidate generation. The fields in the batch determine its publication path: a batch containing only runtime_mutable() fields publishes at the target operator’s next scheduler-safe compute boundary; a batch containing any lifecycle_mutable(stage) field presents its candidate privately to the restart-forward lifecycle callbacks and publishes the whole batch only when the lifecycle transaction reaches kPublished.

The payoff is a very simple mental model for operator authors:

• Each callback reads exactly one coherent, pinned generation. A value never changes underneath a running compute() — there are no torn reads and no mid-callback surprises.
• A later callback of the same operator may observe a newer generation.
• Updates are deterministic and race-free, at the cost of not being instant.

The timeline below shows the runtime-only path: a submit() arriving mid-run reserves a candidate generation, which publishes only at the next compute boundary — where generation 1 begins. It does not describe a lifecycle-mutable or mixed batch; those use the restart path described below.

Inside compute(), read the pinned aggregate through the ExecutionContext:

[[nodiscard]] holoscan::expected<void, holoscan::Error> compute(
    holoscan::ExecutionContext& ctx) override {
  const GainParams& p = ctx.params<GainParams>();  // pinned for this whole callback
  apply_gain(p.gain);
  // ctx.params_generation() reports this callback's generation
  // (0 until an update publishes).
  return {};
}

In start() and stop(), an operator reads the phase-selected generation through Operator::params() instead — ctx.params<T>() is the compute-path accessor. Advanced lifecycle callbacks read the same phase-selected view through their LifecycleContext; the lifecycle callbacks page explains which generation each transition phase presents.

Two kinds of mutable fields

Not every change fits the next-compute boundary. A gain nudge can take effect between two activations, but a pool size or a device binding only makes sense if part of the operator’s lifecycle re-runs. The schema therefore distinguishes two mutability markers, and everything else on this page follows from which one a field carries:

Three rules keep mixed batches coherent:

• One batch, one boundary. A same-operator batch that touches any lifecycle-mutable field becomes a lifecycle candidate as a whole — the runtime-mutable fields in that batch publish with it at the lifecycle barrier, not at next-compute. A runtime-only batch never triggers a restart.
• No leaking ahead. While a lifecycle candidate is pending for an operator, ordinary runtime-update ingress for that operator returns kQueueFull, so a lifecycle-only field can never become visible before the transaction’s successful publication.
• Equivalence can short-circuit. A parameter-only submission whose values equal the published generation may terminate kAcceptedNoChange without restarting — unless a compatible queued transaction already carries explicit restart intent, in which case the submission may join it.

Track a lifecycle-triggered update with the same result types as any other update — the parameter path never returns a LifecycleRestartReceipt. ParamControlHandle::submit returns a ParamBatchUpdateResult with the accepted update sequence, and await(sequence, ...) returns a ParamUpdateResult whose lifecycle_transaction_id and coalesced_into_transaction fields correlate the update with its restart transaction. (The compute-side seam differs again: try_submit_from_compute() returns an initial ParamSubmitResult — see below.)

The opt-in chain: making a field updatable

Updates are opt-in per field, at compile time. A field you do not mark is frozen for the lifetime of the plan, and the plan reserves no update capacity for it. Two links must both be present:

1. The field marker. In the operator’s parameter schema, mark the tunable field runtime_mutable():

   HOLOSCAN_PARAM_SCHEMA(GainParams,
                         param_schema("custom.camera.GainParams", 1),
                         field<&GainParams::gain>("gain").runtime_mutable());

2. The compile-time policy. Enable runtime updates in CompileOptions:

   holoscan::CompileOptions options;
   options.parameters.runtime_updates =
       holoscan::ParamRuntimeUpdatePolicy::kEnabledIfAdmitted;
   const holoscan::ExecutionPlan plan = holoscan::compile(graph, options);

kEnabledIfAdmitted follows the usual compile-and-check style: the compiler enables updates only when every required proof is admitted — including the active profile. This is where bounded-by-construction shows up again: opting in reserves bounded capacity (ingress slots, candidate generation storage, audit records) in the compiled plan, so nothing is allocated on the hot path later.

The chain above is the runtime_mutable() gate. A lifecycle_mutable(stage) field uses a different one: it needs CompileOptions::lifecycle_transactions (see restart control) rather than the runtime-updates policy, and without that admission its submissions report kRequiresLifecycleBarrier — not kLifecycleTransactionsDisabled, which belongs to the dedicated restart APIs.

For the schema builder itself, see Parameter schemas.

Minting a control handle

External application code drives updates through a ParamControlHandle (declared in holoscan/core/parameter_control.hpp), minted from the running session:

holoscan::ParamAccessPolicy policy{
    .permissions = holoscan::ParamPermission::kRead |
                   holoscan::ParamPermission::kUpdate |
                   holoscan::ParamPermission::kAwait,
    .allowed_scope = holoscan::ParamScopeFilter::for_operator(camera_ref),
};
auto minted = session.parameter_control(policy);  // expected<ParamControlHandle, Error>
if (!minted) { /* handle the error */ }
holoscan::ParamControlHandle handle = std::move(minted).value();

The policy is restriction-only: it can only narrow the authority the plan and session already admitted, never widen it. handle.restrict(policy) derives a further-narrowed handle — useful for passing a read-only view to a monitoring component. To find out which fields the handle can actually see and address, consult its policy-filtered discovery catalog via handle.catalog().

Submitting an update

Build a patch addressing the target field and submit it. submit copies the patch and returns synchronously with a typed result:

holoscan::ParamPatch patch{
    .target = holoscan::ParamAddress{.operator_instance = camera_ref,
                                     .field = gain_field},
    .value = holoscan::ParamValue::from(0.5F),
};
holoscan::ParamBatchUpdateResult r =
    handle.submit(patch, holoscan::ParamUpdateOptions{.effective = holoscan::NextCompute{}});
if (!r.committed) {
  // Inspect the rejection category; nothing throws.
}
// A committed result has a retained update_sequence. target_generation is
// the candidate generation, or the unchanged published generation for no-change.

A few details worth knowing:

• Batches are same-operator, all-or-nothing. Pass std::span<const holoscan::ParamPatch> to update several fields of one operator atomically. A batch spanning multiple operators rejects with kUnsupportedAtomicScope.
• Optimistic concurrency. Set patch.expected_generation to make the update conditional on the latest accepted generation; a mismatch rejects with kExpectedGenerationMismatch.
• Equivalent values short-circuit. Submitting a value equal to the current one can return kAcceptedNoChange — accepted, but nothing publishes.

Observing an update

Three complementary tools cover “has it landed?”, “what would compute see now?”, and “tell me when anything changes.”

Await a terminal state. await blocks (with a timeout) until the accepted sequence reaches any terminal state. kPublished is terminal for update processing, and a later activation may advance the retained state to kObserved once a compute() has actually pinned the new generation:

const auto done = handle.await(r.update_sequence, std::chrono::seconds{1});
if (!done) {
  if (done.error().code == holoscan::ErrorCode::kTimeout) {
    // Still pending. A timeout never cancels the update; await it again later.
  } else {
    // Handle permission, stale-handle, closed-session, or argument errors.
  }
} else {
  switch (done->state) {
    case holoscan::ParamUpdateState::kPublished:
    case holoscan::ParamUpdateState::kObserved:
      // Publication succeeded. kObserved also proves a callback pinned it.
      break;
    case holoscan::ParamUpdateState::kAcceptedNoChange:
      // The value was equivalent; no new generation was published.
      break;
    case holoscan::ParamUpdateState::kSuperseded:
    case holoscan::ParamUpdateState::kCanceled:
      // This accepted sequence ended without publication.
      break;
    case holoscan::ParamUpdateState::kRolledBack:
      // Lifecycle publication failed; the previous generation was restored.
      break;
    case holoscan::ParamUpdateState::kFailedStopped:
      // Recovery failed and the affected run stopped.
      break;
    default:
      // await() does not return a nonterminal state for an accepted sequence.
      break;
  }
}

An immediate submit rejection has no accepted sequence and cannot be awaited. kAcceptedNoChange is terminal but is not a publication event.

Read the current view. read returns a coherent record without waiting:

if (auto rec = handle.read(patch.target, holoscan::ParamReadView::kPublished)) {
  const float live = rec->value.as<float>().value_or(0.0F);
  // rec->generation is the generation this value belongs to.
}

ParamReadView::kPublished is the value a future compute() would see right now; kLatestAccepted includes a pending-but-unpublished candidate; and kPending returns only an unpublished candidate (absence reports as not ready). For a broader coherent view across fields, handle.capture(...) — or session.capture_params(...) from the session — returns an owning ParamSnapshot.

Follow the event stream. For change notifications, open a bounded per-capability event cursor (the handle needs ParamPermission::kObserve) and drain it from cold or control code:

if (auto cursor = handle.open_event_cursor()) {
  std::array<holoscan::ParamEvent, 16> events{};
  if (auto n = cursor->drain(events)) { /* process events[0..*n) */ }
}

The cursor carries transition records, never parameter values. If retention overflows, the loss surfaces as a single explicit ParamEventKind::kGap record — resynchronize with a read or a capture rather than assuming continuity.

Submitting from inside compute()

An operator may want to self-tune — for example, a fusion operator adjusting its own blend weight based on observed input staleness.

Submit a general lifecycle_mutable(stage) patch from external or control context with ParamControlHandle::submit. A compute-origin lifecycle patch can reach the lifecycle transaction path only when the same plan separately admits runtime updates and freezes positive compute-ingress and per-request patch capacity. A lifecycle-only plan reserves neither; a one-patch compute submission returns kOutOfBounds.

The compute seam is reached through the ExecutionContext:

[[nodiscard]] holoscan::expected<void, holoscan::Error> compute(
    holoscan::ExecutionContext& ctx) override {
  const holoscan::ParamInlinePatch patch{
      .target = holoscan::ParamAddress{.operator_instance = ctx.self(),
                                       .field = gain_field},
      .value = holoscan::ParamValue::from(new_gain),
  };
  const holoscan::ParamSubmitResult sr =
      ctx.param_control().try_submit_from_compute({&patch, 1});
  // sr.status == kSubmitted on enqueue; kQueueFull rejects this intent.
  // This minimal example drops rejected intents; it does not arrange a retry.
  return {};
}

Two behaviors distinguish the seam from the owning handle:

• A successful enqueue returns kSubmitted with sequence zero — identity is assigned only after the bounded ingress drains and the update is accepted. There is nothing to await from inside the callback.
• The submission never changes the generation pinned by the current callback. After bounded ingress drains, a runtime-only batch publishes at a future compute boundary; a batch touching a lifecycle-mutable field instead follows the lifecycle transaction path. Either way, only a later callback can observe a different generation.

kQueueFull can mean either full bounded ingress or transient ingress-lock contention. In either case, this attempt enqueued no patches. To retry, keep the desired values in bounded operator-owned state or reconstruct them, and explicitly resubmit from a later callback that the application’s trigger will actually produce. Runtime does not retain a rejected intent, schedule a retry callback, or replay compute(). If no later callback is guaranteed, “retry later” is not a delivery guarantee. This differs from retrying a publication that consumed a Tensor loan or may already have reached a recipient; see backpressure and callback retry.

Fencing further updates

When a run must stop accepting parameter changes — a validated configuration entering a measurement window, for example — close the update ingress:

auto fence = session.close_parameter_update_ingress();
// expected<ParamStopFenceResult, Error>

The fence is irreversible for the run: no later submission is accepted through either path, from the owning handle or from inside compute(). Updates already accepted before the fence still complete their publication. The result records the final sequence accepted before closure, so control code can await everything that made it in.

Recap

• Parameters live in immutable versioned generations; generation 0 is the compile-resolved initial aggregate, and each callback pins exactly one coherent snapshot — no torn reads, ever.
• A runtime-only field is updatable only through the full opt-in chain: runtime_mutable() on the field and ParamRuntimeUpdatePolicy::kEnabledIfAdmitted in CompileOptions, under the Standard or Research profile.
• The two mutability markers publish differently: runtime_mutable() swaps generations at the target’s next compute boundary; lifecycle_mutable(stage) routes the whole batch through a lifecycle restart transaction gated by CompileOptions::lifecycle_transactions under the Standard or Research profile, publishes after kPublished, and correlates via ParamUpdateResult::lifecycle_transaction_id.
• session.parameter_control(policy) mints a restriction-only ParamControlHandle; submit returns a typed result, and acceptance is a reservation. await returns any terminal result; treat the value as published only when state is kPublished or kObserved.
• Operator code self-updating runtime_mutable() fields uses ctx.param_control().try_submit_from_compute(...); general lifecycle_mutable(stage) updates use an external/control-context owning handle, which is forbidden from callback contexts.
• close_parameter_update_ingress() irreversibly fences the run against further updates.

Next: Determinism and replay builds on the same scheduler-boundary discipline to make whole runs reproducible.