Running and Stopping

Compilation gave you an accepted ExecutionPlan — an immutable, self-contained description of what to run. This page covers the third phase of the compose → compile → run model: turning that plan into a running session, watching it, stopping it cooperatively, and finding out why it ended.

There are two execution families. run(...) blocks until the run reaches a terminal state, while run_async(...) returns immediately with a RunSession that owns everything the run started. C++ exposes explicit managed-signal policies; Python deliberately leaves process signals with the caller. There is no scheduler or executor object to construct or configure.

For startup barriers, a scoped restart inside the same session, and the difference from reconstructing a new session, see Lifecycle and Restart in Depth.

Complete C++ example: examples/running_and_stopping/cpp. The Python tabs below show the corresponding session API.

The blocking entry point: run()

When your main thread has nothing else to do while the pipeline executes, call run(). It blocks until the run reaches a terminal state:

#include <holoscan/core/run.hpp>
const holoscan::ExecutionPlan plan = holoscan::compile(graph);
if (!plan.ok()) {
  return 1;
}
const int rc = holoscan::run(plan);

In C++, rc == 0 means the run ended successfully: either natural finite completion (RunTermination::kCompleted) or explicit operator-requested completion (RunTermination::kRequestedCompleted). A compiler-accepted empty Graph is a valid zero-work plan and completes naturally. A nonempty graph does not infer completion from a temporarily empty queue. Eligible finite sources can declare complete_after(ComputeCalls{N}): after N successful callbacks their outputs close, EOS propagates, and downstream work drains before kCompleted. An active unbounded source prevents natural whole-graph completion. Any nonzero result means the run was stopped, failed, or rejected during managed-signal admission before session creation. run() is [[nodiscard]] — the compiler will remind you not to drop the result.

On Linux, blocking run(plan) manages process-directed SIGINT and SIGTERM by default. The first synchronously accepted occurrence requests ordinary cooperative stop and returns nonzero after cleanup. This is equivalent to run(plan, RunSignalPolicy::kManaged).

Choose signal ownership deliberately. The following table describes the C++ overloads; Python run() and run_async() are always caller-managed:

An embedding process that already owns process signals can opt out while keeping the convenient blocking result:

const int rc = holoscan::run(
    plan, holoscan::RunSignalPolicy::kCallerManaged);

The caller-managed blocking overload performs no managed owner acquisition, thread census, signal-mask change, descriptor setup, terminal participation, or managed-signal diagnostic publication. It is the direct replacement for an embedding host that previously used run(plan) with its own handlers or signal-waiting thread. Plain async is also caller-managed:

holoscan::RunSession session = holoscan::run_async(plan);
// Application-owned control code may call session.request_stop().
session.wait();

Do not try to make default managed run(plan) work by changing only one thread in an already-multithreaded process. Signal masks are per-thread, and that does not establish process-wide ownership. With no adoptable domain, successful census reads that change or either observe anything other than exactly one live task report RUNTIME_MANAGED_SIGNAL_DOMAIN_REQUIRED; a census read error reports RUNTIME_MANAGED_SIGNAL_INSTALL_FAILED. Admission returns nonzero before session creation in either case. In the originating process, an existing unavailable domain or binding, or a poisoned process registry, instead reports RUNTIME_MANAGED_SIGNAL_OWNER_BUSY. Let an active binding retire; close and recreate a spent domain; reconcile an ownership conflict or choose caller-managed execution. After registry poisoning or another irreparable cleanup failure, contain and restart the process rather than retrying.

Managed signals with run_async()

For managed asynchronous execution, create the process domain near the start of main(), before graph, CUDA, logger, framework, or application setup that may create participant threads. Then bind the session as part of the run_async call; attaching to a session that has already started would race a fast terminal run and is intentionally not supported.

#include <iostream>
#include <utility>
#include <holoscan/core/managed_shutdown_signals.hpp>
#include <holoscan/core/run.hpp>
auto created = holoscan::ManagedShutdownSignals::create();
if (!created) {
  std::cerr << holoscan::runtime_error_message(created.error()) << '\n';
  return 1;
}
auto signals = std::move(created).value();
holoscan::RunSession session = holoscan::run_async(plan, signals);
session.wait();
auto closed = signals.close();
if (!closed) {
  std::cerr << holoscan::runtime_error_message(closed.error()) << '\n';
  return 1;
}

ManagedShutdownSignals is move-only, creator-thread-affine, and one-shot. Create, bind, and close it on its creator thread; keep it alive until its bound session and every participant thread have retired. session.wait() includes managed binding retirement, so an immediate creator-thread signals.close() is deterministic. A domain that is available or spent but currently unbound forwards accepted signals under the unchanged prior policy; it never redirects them to a caller-managed blocking run or plain async run. Plain run(plan) may adopt an available explicit domain. That successful reservation consumes the domain’s sole binding attempt, so after retirement close the spent domain and create a fresh one before another managed run.

On non-Linux platforms, ManagedShutdownSignals::create() returns ErrorCode::kNotSupported, while both blocking policy values use ordinary blocking execution. Active-domain HSDK use after fork(), adoption of arbitrary pre-existing threads, and signals directed with pthread_kill() to another participant thread are currently unsupported.

The child of fork() inherits the creator’s blocked SIGINT/SIGTERM mask, and exec() preserves that mask even though the managed descriptors are close-on-exec. In the post-fork child, reset the mask with async-signal-safe sigprocmask() before exec(), or use posix_spawn() with POSIX_SPAWN_SETSIGMASK and an empty or application-chosen mask. system() and popen() do not reset the inherited mask for you.

First and second accepted signals

HSDK installs no asynchronous signal handler and does not mutate sigaction. Every participant in a managed domain inherits blocked SIGINT/SIGTERM, and an ordinary controller or broker synchronously accepts process-directed occurrences with Linux signalfd:

1. The first accepted occurrence publishes RUNTIME_MANAGED_SIGNAL_RECEIVED and requests kStopped cleanup at most once.
2. A second accepted occurrence publishes RUNTIME_MANAGED_SIGNAL_ESCALATED, disarms management, and delegates that actual second signal under the unchanged default, ignored, or custom prior policy.

An unchanged custom disposition does not promise delivery-thread affinity. For a persistent domain, an unblocked custom handler may run on the broker thread both when an unbound domain forwards a signal and when a bound session delegates the second accepted signal. A handler must not assume the creator thread’s TLS, affinity, or identity.

SIGINT and SIGTERM are standard, non-real-time signals. Identical occurrences can coalesce while pending, and the order of simultaneously pending mixed signals is unspecified. A deterministic supervisor that wants to exercise the second-signal path must wait until first-acceptance evidence is observable before sending the second signal. “Second” means second synchronously accepted occurrence, not necessarily second generated occurrence.

Signal handling remains cooperative. A first signal adds no forced cancellation and no whole-run completion bound: run() can still wait indefinitely if user code ignores cancellation and never returns.

Managed-signal diagnostics use seven stable identifiers:

An irreparable wait, notification, disposition, delegation, restoration, or cleanup failure poisons managed admission so a later caller cannot assume clean process state. Complete application-level containment and restart rather than retrying managed ownership in that process.

run() executes the same session path as run_async() while its selected policy controls only the surrounding signal domain. The blocking int cannot distinguish stop, execution failure, and managed-domain admission failure. When you need a typed setup error, establish ManagedShutdownSignals explicitly; when you need to control the session or inspect terminal diagnostics, use a run_async() overload as shown below.

Choose who presents failure diagnostics

Automatic Core reports are enabled by default. For an admitted plan whose startup or compute fails, this is sufficient to receive its existing failure summary after cleanup; no setter or helper is required:

// plan is an admitted application plan.
const int result = holoscan::run(plan, holoscan::RunSignalPolicy::kCallerManaged);

set_automatic_diagnostic_logging_enabled(true) is normally redundant. It re-enables future reports after API opt-out, without immediately printing or replaying anything. It never causes compile() to dump its returned records. For caller-owned terminal reporting, retain the session and report after waiting:

#include <holoscan/core/diagnostic.hpp>
#include <holoscan/core/run.hpp>
holoscan::set_automatic_diagnostic_logging_enabled(false);
auto session = holoscan::run_async(plan);
session.wait();
holoscan::log_diagnostics(session.diagnostics(), plan.graph());

The explicit helper works with automatic reporting on or off. A valid HOLOSCAN_AUTOMATIC_DIAGNOSTIC_LOGGING=ON environment override wins over API false; use the getter to check the effective policy. Keeping automatic reports enabled and explicitly reporting the same terminal records can duplicate output. The helper neither waits nor consumes diagnostics. An immediate async prefix may be incomplete because async return is not a startup barrier. Blocking run() returns an exit code, so use run_async() when you need to retain and present terminal records yourself. Normal logger filtering applies to both presentation paths; failed outcomes and structured sinks remain intact.

Fatal self-join and internal invariant violations are different from an ordinary failed session. They attempt their original ERROR message independently of the automatic-report setting, then call std::terminate(). Normal logger filtering still applies, and logging exceptions cannot prevent termination. There is no successful return from wait() and no later opportunity to replay diagnostics from an aborted process.

Finite sources: close outputs and drain

An eligible source has no input ports, at least one output, and a supported source trigger. Add complete_after(holoscan::ComputeCalls{.limit = N}) to its C++ contract, or .complete_after(N) in Python. The rule provides no trigger: the source still chooses OnClock or another admitted activation mechanism.

After N successful compute calls, the source closes its outputs through EOS. Its final accepted publications and downstream work drain before natural kCompleted / Python COMPLETED. Zero closes without a compute call. An active unbounded source prevents natural whole-graph completion. A callback that emits nothing but returns success still consumes a call; publication retries therefore need separate message accounting. Inputful operators cannot use this rule.

request_completion() instead requests immediate whole-run success, with no EOS or downstream-drain promise. A sink count can define a local stopping condition but cannot establish that other branches drained.

The asynchronous entry point: run_async() and RunSession

run_async() starts the run and immediately returns a RunSession that owns the run and every activity it started:

#include <holoscan/core/run_session.hpp>
holoscan::RunSession session = holoscan::run_async(plan);
// Your thread is free: serve a UI, run the application's caller-managed
// signal/control loop, or do other work.
// The source graph and even your ExecutionPlan wrapper may be destroyed now —
// the session reconstructed everything it needs from the plan.

The plan is self-contained: the session rebuilds fresh operators and clocks from the plan’s own factories, so repeated runs never alias each other’s state and the authoring objects need not outlive the run.

Strict RAII: the session is the run

RunSession follows a small, strict set of lifetime rules:

• Move-only. There is no copy and no move-assignment. Move-construction transfers the entire owner; the moved-from wrapper becomes inert, reports kStopped, and its operations become no-ops.
• The destructor stops and joins. Destroying a live session requests a cooperative stop and then joins every owned activity. It never detaches, throws, or force-kills user code. When the wrapper is gone, the run is done — ownership is unambiguous by construction.
• The destructor can block. Because it joins rather than kills, the destructor waits for your operators’ compute() calls to return. Make compute() return promptly when the runtime signals cancellation.

Stopping and waiting

An operator can end its own run without holding a RunSession. Call context.request_completion() from compute() when its work means the run finished successfully, or context.request_stop() when it is cancelling the run cooperatively. Both return immediately. The accepted request closes callback admission at one session-wide publication point: callback claims ordered before it may proceed and retire, but no claim ordered afterward enters.

holoscan::expected<void, holoscan::Error> compute(holoscan::ExecutionContext& context) override {
  if (finished_work()) {
    context.request_completion();
  }
  return {};
}

Completion reports the distinct success RunTermination::kRequestedCompleted and makes run(plan) return zero. Stop reports RunTermination::kStopped and makes run(plan) return nonzero. Repeated requests are idempotent; a stop request promotes and overrides completion. A callback, runtime, or cleanup failure overrides both and reports kFailed, except for the stop-caused publication failure described in publications blocked at stop.

Stopping is a two-step, cooperative protocol:

session.request_stop();  // ask: thread-safe, nonblocking, idempotent
session.wait();          // block until every owned activity has joined

request_stop() only asks: it publishes the cancellation signal and wakes the run’s worker, then returns immediately. Diagnostics publication and cleanup remain owned by the run itself. It is safe to call from any external thread, safe to call twice, and a no-op on a moved-from wrapper. To actually block until the run has finished, call wait(). Like the destructor, wait() may block indefinitely if an operator ignores cancellation and never returns from compute().

A direct application request_stop() normally reaches kStopped without a managed-signal diagnostic. A first synchronously accepted managed signal also reaches kStopped, but retains a warning-level RUNTIME_MANAGED_SIGNAL_RECEIVED record. A stopped run can also carry warning-level RUNTIME_OPERATOR_PUBLICATION_STOPPED records (see below). Do not use diagnostics().empty() to classify every stopped run; inspect the stable token when the origin of a stop matters.

Publications blocked at stop

An accepted stop closes callback admission, so a downstream operator whose input queue is full may never run again. A blocking publication that is waiting for such a recipient when the stop arrives, or starts waiting after it, does not wait out the graph blocking budget: it returns at once with its ordinary publication-pressure code. That code is kBackpressured, or kPublicationPartial or kPublicationIndeterminate when other recipients of the same fanout already accepted the sample. The error message begins with RUNTIME_PUBLICATION_STOPPED. Publications to recipients with room still succeed.

If the callback returns that error unchanged, for example through HOLOSCAN_TRY, the run still ends kStopped rather than kFailed. diagnostics() then carries one warning-level RUNTIME_OPERATOR_PUBLICATION_STOPPED record for the operator, with the original error as its cause. The exception is narrow:

• The stop must have cut one of that callback’s own publications short: the publication was waiting for a full recipient when the stop was accepted, or started waiting after it. Returning a pressure code without such a publication still fails the run.
• The callback must return that publication’s own error, with its code and message unchanged. After handling such an error, returning another publication’s error, a new error, or a changed copy still fails the run.
• A publication whose wait ran out its budget before the stop, a publication that failed for another reason (for example a transport error after other recipients accepted), and every other callback error still fail the run.
• So does a failure to retire that callback’s activation, or a failure anywhere else in the run.
• Requested completion does not qualify: a publication that fails while the run completes still fails it.
• The exception applies to ordinary operator callbacks. Members of a closed-island composite and fused CUDA Graph members keep the failure classification.

To stop without the warning, check for cancellation after a failed publication and return success, as the graph-composition example does. This fragment runs inside compute(); output_ and frame are the operator’s output port and sample:

auto published = output_.emit(std::move(frame));
if (!published) {
  if (context.cancellation().stop_requested()) {
    return {};  // stopping: this sample is not delivered
  }
  return holoscan::make_unexpected(std::move(published).error());
}
return {};

Query operations — is_running(), termination(), diagnostics(), request_stop(), and wait() — may be called concurrently on one session. Moving or destroying the wrapper is the exception: that requires external exclusion from every other operation on it.

Reading the outcome: termination()

termination() reports the session’s coarse outcome. It returns kRunning until the run reaches one stable terminal state, then never changes:

switch (session.termination()) {
  case holoscan::RunTermination::kCompleted:
    // Natural finite completion.
    break;
  case holoscan::RunTermination::kRequestedCompleted:
    // Explicit immediate whole-run success; no downstream-drain guarantee.
    break;
  case holoscan::RunTermination::kStopped:
    // A cooperative stop completed (or this wrapper is moved-from).
    break;
  case holoscan::RunTermination::kFailed:
    // Startup or execution failed; see diagnostics() below.
    break;
  case holoscan::RunTermination::kRunning:
    // Work is still outstanding — not terminal yet.
    break;
}

is_running() stays true through startup, execution, the stop-requested window, and cleanup; treat it as “work is still outstanding,” not as “compute is currently executing.”

Reading diagnostics() on failure

A kFailed session is not a dead end: the session retains a bounded set of structural Diagnostic records explaining why the run stopped or failed. Each record carries a stable registry token — a fixed spelling such as RUNTIME_CLOCK_BINDING_UNAVAILABLE, not free-form log prose — plus an optional plan-local source location and optional owning failure detail:

if (session.termination() == holoscan::RunTermination::kFailed) {
  for (const holoscan::Diagnostic& d : session.diagnostics()) {
    std::cerr << holoscan::format_diagnostic(d, plan.graph()) << '\n';
  }
}

The normal failed-session report already prints the retained explanation at ERROR level. Explicit application formatting is useful for a custom sink or UI, but may duplicate that output. Callback detail includes the qualified operator path, original message, and lifecycle phase; inspect d.detail directly when program logic needs those fields. Supplemental detail is not canonical plan JSON. Clean completion and clean cooperative stop produce no failure report.

The returned span is append-only and stable: the storage and the prefix you observed remain valid until the wrapper is moved or destroyed, so you may hold the span safely. Before the session is terminal, a later diagnostics() call may expose a longer prefix — reacquire it rather than assuming the one you saw was complete. For how to interpret the tokens themselves, see Understanding failures.

The execution and signal-domain failure channels

Run-phase outcomes and explicit signal-domain administration use distinct channels; do not collapse them:

1. Exceptions at the boundary. run() and run_async() throw holoscan::CompileError if the plan is a default-constructed/no-state wrapper or was rejected at compile admission — so check plan.ok() first, or be prepared to catch. Beyond that, only std::system_error from a failed platform thread join can escape run() or wait(). That is the entire set.
2. A terminal kFailed session. Once a session is created, failures — a required runtime binding that is unavailable, a worker that fails to start, resource exhaustion during startup, an execution failure — is reported as termination() == kFailed (a nonzero run() result) plus diagnostics() records. No exception is thrown for these.
3. Typed explicit-domain administration. ManagedShutdownSignals::create() and close() return expected<..., Error>. Setup, wrong-thread, ownership, platform, and close/restoration failures are values rather than exceptions. A managed run_async(plan, signals) binding-admission failure returns a terminal failed session with bounded managed-signal diagnostics. Default blocking managed admission is the explicit pre-session exception: it has only its int result and returns nonzero before session creation. The blocking API exposes bounded terminal/log evidence, but not a typed admission result. Use explicit-domain creation for typed setup errors and managed async execution for inspectable session diagnostics.

This split is deliberate and mirrors the failure styles of the three-phase model: authoring throws, compile() returns rejected plans, and run fails closed. An accepted plan whose required runtime bindings are unavailable terminates kFailed before any user callback runs, rather than silently degrading to weaker semantics.

Restarting operators

A run is not all-or-nothing: when the plan grants the authority, individual operators can be restarted in place while the rest of the graph keeps running. That machinery has its own page — see Restart control.

Recap

• C++ run(plan) blocks and returns an int; on Linux it uses managed SIGINT and SIGTERM by default. Python run(plan) is caller-managed and returns the terminal session. Eligible finite sources declare complete_after and close through EOS so downstream drains before natural completion; empty queues alone are not completion evidence.
• Use run(plan, RunSignalPolicy::kCallerManaged) when the host owns process signals. It performs no HSDK-managed signal-domain work. Plain run_async(plan) is likewise caller-managed on every platform.
• For managed async, create one move-only ManagedShutdownSignals domain near main() entry, call run_async(plan, signals), wait for the session, then close the one-shot domain on its creator thread.
• The first synchronously accepted process signal requests cooperative stop; the second accepted signal is delegated under the unchanged prior policy. Standard signals may coalesce, and simultaneously pending mixed order is unspecified. HSDK installs no signal handler and mutates no disposition.
• An operator may call request_completion() for immediate whole-run success, without EOS, source-closure, queued-delivery, or downstream-drain guarantees.
• C++ run_async(plan) returns a move-only RunSession; Python returns a context manager. Destruction or context exit requests stop and joins everything the run started.
• Stop cooperatively: request_stop() asks and returns immediately; wait() blocks until the run has fully joined.
• Never call wait() or destroy the session from run-owned code — that is a fatal contract violation.
• Branch on termination() (kCompleted / kRequestedCompleted / kStopped / kFailed / kRunning); on failure, read the stable tokens in session.diagnostics().
• Execution uses CompileError (plus std::system_error on join) at the boundary and terminal kFailed sessions after admission. Explicit domain create/close adds a typed expected<..., Error> administration channel.

Next: wire these pieces into a complete application in Your first pipeline.