> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/holoscan/sdk-user-guide/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/holoscan/sdk-user-guide/_mcp/server.

# holoscan::GPUResidentExecutor

```cpp showLineNumbers={false}
#include <holoscan/gpu_resident_executor.hpp>
```

**Inherits from:** `holoscan::Executor` (public)

---

## Constructors

### GPUResidentExecutor \[#gpuresidentexecutor]

#### From raw pointer

```cpp showLineNumbers={false}
holoscan::GPUResidentExecutor::GPUResidentExecutor(
    Fragment *fragment
)
```

Construct a new `GPUResidentExecutor` object.

**Parameters**

**`fragment`** `Fragment *`

The pointer to the fragment of the executor.

---

#### Deleted overload 1

```cpp showLineNumbers={false}
holoscan::GPUResidentExecutor::GPUResidentExecutor() = delete
```

### Destructor \[#destructor]

### \~GPUResidentExecutor

```cpp showLineNumbers={false}
holoscan::GPUResidentExecutor::~GPUResidentExecutor()
```

---

## Methods

### run \[#run]

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::run(
    OperatorFlowGraph &graph
) override
```

Run the graph.

**Parameters**

**`graph`** `OperatorFlowGraph &`

The reference to the graph.

---

### run\_async \[#runasync]

```cpp showLineNumbers={false}
std::future<void> holoscan::GPUResidentExecutor::run_async(
    OperatorFlowGraph &graph
) override
```

Run the graph asynchronously.

**Returns:** The future object.

**Parameters**

**`graph`** `OperatorFlowGraph &`

The reference to the graph.

---

### context \[#context]

#### Set the context

virtual

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::context(
    void *context
) override
```

Set the context.

**Parameters**

**`context`** `void *`

The context.

---

#### Get the context

```cpp showLineNumbers={false}
void * holoscan::GPUResidentExecutor::context()
```

Get the context.

**Returns:** The context.

### initialize\_fragment \[#initializefragment]

```cpp showLineNumbers={false}
bool holoscan::GPUResidentExecutor::initialize_fragment() override
```

Initialize the fragment\_ in this [Executor](executor).

This method is called by run() to initialize the fragment and the graph of operators in the fragment before execution.

**Returns:** true if fragment initialization is successful. Otherwise, false.

### initialize\_operator \[#initializeoperator]

```cpp showLineNumbers={false}
bool holoscan::GPUResidentExecutor::initialize_operator(
    Operator *op
) override
```

Initialize the given operator.

This method is called by Operator::initialize() to initialize the operator.

Depending on the type of the operator, this method may be overridden to initialize the operator. For example, the default executor (GXFExecutor) initializes the operator using the GXF API and sets the operator's ID to the ID of the GXF codelet.

**Returns:** true if the operator is initialized successfully. Otherwise, false.

**Parameters**

**`op`** `Operator *`

The pointer to the operator.

---

### initialize\_scheduler \[#initializescheduler]

```cpp showLineNumbers={false}
bool holoscan::GPUResidentExecutor::initialize_scheduler(
    Scheduler *sch
) override
```

Initialize the given scheduler.

This method is called by Scheduler::initialize() to initialize the operator.

Depending on the type of the scheduler, this method may be overridden to initialize the scheduler. For example, the default executor (GXFExecutor) initializes the scheduler using the GXF API and sets the operator's ID to the ID of the GXF scheduler.

**Returns:** true if the scheduler is initialized successfully. Otherwise, false.

**Parameters**

**`sch`** `Scheduler *`

The pointer to the scheduler.

---

### initialize\_network\_context \[#initializenetworkcontext]

```cpp showLineNumbers={false}
bool holoscan::GPUResidentExecutor::initialize_network_context(
    NetworkContext *network_context
) override
```

Initialize the given network context.

This method is called by NetworkContext::initialize() to initialize the operator.

Depending on the type of the network context, this method may be overridden to initialize the network context. For example, the default executor (GXFExecutor) initializes the network context using the GXF API and sets the operator's ID to the ID of the GXF network context.

**Returns:** true if the network context is initialized successfully. Otherwise, false.

**Parameters**

**`network_context`** `NetworkContext *`

The pointer to the network context.

---

### initialize\_fragment\_services \[#initializefragmentservices]

```cpp showLineNumbers={false}
bool holoscan::GPUResidentExecutor::initialize_fragment_services() override
```

Initialize the fragment services for the executor.

This method is called during executor initialization to set up any required fragment services.

Depending on the type of executor, this method may be overridden to initialize specific fragment services. For example, the default executor (GXFExecutor) may initialize fragment services using the GXF API.

**Returns:** true if the fragment services are initialized successfully. Otherwise, false.

### prepare\_data\_flow \[#preparedataflow]

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::prepare_data_flow(
    const std::shared_ptr<OperatorFlowGraph> &graph,
    const std::vector<std::shared_ptr<Operator>> &topo_ordered_operators
)
```

Prepare data flow connections for a topologically ordered GPU-resident graph.

This initializes operator specs, assigns per-port unique IDs, and allocates/connects device memory for every supported edge in the graph.

**Parameters**

**`graph`** `const std::shared_ptr<OperatorFlowGraph> &`

The operator graph.

---

**`topo_ordered_operators`** `const std::vector<std::shared_ptr<Operator>> &`

Operators flattened in deterministic topological order.

---

### initialize\_cuda \[#initializecuda]

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::initialize_cuda()
```

This function initializes CUDA.

Currently, it sets the device to 0 by default. Setting a different GPU device for GPU-resident graph execution is not yet supported.

### device\_memory \[#devicememory]

```cpp showLineNumbers={false}
void * holoscan::GPUResidentExecutor::device_memory(
    const std::shared_ptr<Operator> &op,
    const std::string &port_name
)
```

This function returns the device memory address of an input or output port corresponding to a given port name.

GPU-resident operators use this function to get the device memory address of the input or output port.

**Returns:** The device memory address of the input or output port

**Parameters**

**`op`** `const std::shared_ptr<Operator> &`

The operator

---

**`port_name`** `const std::string &`

The name of the input or output port

---

### verify\_graph\_topology \[#verifygraphtopology]

```cpp showLineNumbers={false}
virtual bool holoscan::GPUResidentExecutor::verify_graph_topology(
    const std::shared_ptr<OperatorFlowGraph> &graph,
    std::vector<std::shared_ptr<Operator>> &topo_ordered_operators
)
```

Verify the graph topology and flatten it in topological order.

GPU-resident execution currently supports acyclic graphs with exactly one source operator. This method only validates and flattens the operator graph itself.

**Returns:** True if the graph topology is supported by GPU-resident execution, false otherwise.

**Parameters**

**`graph`** `const std::shared_ptr<OperatorFlowGraph> &`

The operator graph.

---

**`topo_ordered_operators`** `std::vector<std::shared_ptr<Operator>> &`

Output vector populated in deterministic topological order.

---

### timeout\_ms \[#timeoutms]

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::timeout_ms(
    unsigned long long timeout_ms
)
```

### tear\_down \[#teardown]

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::tear_down()
```

Sends a tear down signal to the GPU-resident CUDA graph.

### result\_ready \[#resultready]

```cpp showLineNumbers={false}
bool holoscan::GPUResidentExecutor::result_ready()
```

Indicates whether the result of a single iteration of the GPU-resident CUDA graph is ready or not.

**Returns:** true if the result is ready, false otherwise.

### data\_ready \[#dataready]

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::data_ready()
```

This function informs GPU-resident CUDA graph that the data is ready for the main workload.

### is\_launched \[#islaunched]

```cpp showLineNumbers={false}
bool holoscan::GPUResidentExecutor::is_launched()
```

Indicates whether the GPU-resident CUDA graph has been launched.

**Returns:** true if the CUDA graph has been launched, false otherwise.

### execution\_context \[#executioncontext]

```cpp showLineNumbers={false}
std::shared_ptr<ExecutionContext> holoscan::GPUResidentExecutor::execution_context()
```

Get the execution context - currently, this has no meaning for GPU-resident graph execution When we need to store something for execution context, we will store a pointer in the exec\_context\_ for a [ExecutionContext](executioncontext) object.

### graph\_capture\_stream \[#graphcapturestream]

```cpp showLineNumbers={false}
std::shared_ptr<cudaStream_t> holoscan::GPUResidentExecutor::graph_capture_stream()
```

### data\_ready\_handler\_capture\_stream \[#datareadyhandlercapturestream]

```cpp showLineNumbers={false}
std::shared_ptr<cudaStream_t> holoscan::GPUResidentExecutor::data_ready_handler_capture_stream()
```

### workload\_graph\_clone \[#workloadgraphclone]

```cpp showLineNumbers={false}
cudaGraph_t holoscan::GPUResidentExecutor::workload_graph_clone() const
```

### data\_ready\_device\_address \[#datareadydeviceaddress]

```cpp showLineNumbers={false}
void * holoscan::GPUResidentExecutor::data_ready_device_address()
```

Get the CUDA device pointer for the data\_ready signal.

**Returns:** Pointer to the device memory location for data\_ready signal.

### result\_ready\_device\_address \[#resultreadydeviceaddress]

```cpp showLineNumbers={false}
void * holoscan::GPUResidentExecutor::result_ready_device_address()
```

Get the CUDA device pointer for the result\_ready signal.

**Returns:** Pointer to the device memory location for result\_ready signal.

### tear\_down\_device\_address \[#teardowndeviceaddress]

```cpp showLineNumbers={false}
void * holoscan::GPUResidentExecutor::tear_down_device_address()
```

Get the CUDA device pointer for the tear\_down signal.

**Returns:** Pointer to the device memory location for tear\_down signal.

### data\_ready\_handler \[#datareadyhandler]

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::data_ready_handler(
    std::shared_ptr<Fragment> fragment
)
```

Register a data ready handler fragment.

This function stores a reference to the fragment that will handle data ready events.

**Parameters**

**`fragment`** `std::shared_ptr<Fragment>`

The fragment to register as the data ready handler.

---

### data\_ready\_handler\_fragment \[#datareadyhandlerfragment]

```cpp showLineNumbers={false}
std::shared_ptr<Fragment> holoscan::GPUResidentExecutor::data_ready_handler_fragment()
```

Get the registered data ready handler fragment.

**Returns:** The data ready handler fragment, or nullptr if none is registered.

### data\_not\_ready\_sleep\_interval\_us \[#datanotreadysleepintervalus]

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::data_not_ready_sleep_interval_us(
    unsigned int sleep_interval_us = 500
)
```

Set the sleep interval on device when data is not ready.

**Parameters**

**`sleep_interval_us`** `unsigned int` — default: 500

The sleep interval in microseconds. Default is 500 us.

---

### sync\_with\_host \[#syncwithhost]

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::sync_with_host(
    bool enable
)
```

Enable or disable a system-wide fence in the while-end-marker kernel.

**Parameters**

**`enable`** `bool`

True to enable, false to disable.

---

**See also:**
Fragment::GPUResidentAccessor::sync\_with\_host for the public-facing API and full documentation.

### enable\_perf\_measurement \[#enableperfmeasurement]

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::enable_perf_measurement(
    unsigned int num_samples = 100
)
```

Enable execution time measurement.

Execution time is the time between the start of a streaming data iteration and the end of the same iteration. Execution time is not measured when the data is not marked as ready.

**Parameters**

**`num_samples`** `unsigned int` — default: 100

The total number of samples to collect. Default is 100.

---

### execution\_times\_us \[#executiontimesus]

```cpp showLineNumbers={false}
std::pair<unsigned int *, unsigned int> holoscan::GPUResidentExecutor::execution_times_us()
```

Get the host pointer to the execution times in microseconds.

**Returns:** a pair of the host pointer to the execution times in microseconds and the number of samples collected.

### interrupt \[#interrupt]

```cpp showLineNumbers={false}
virtual bool holoscan::GPUResidentExecutor::interrupt()
```

Interrupt the execution.

**Returns:** true if the interrupt was successful (graph was running), false if the graph was not running (already stopped or not started).

### wait \[#wait]

```cpp showLineNumbers={false}
virtual void holoscan::GPUResidentExecutor::wait()
```

Wait for the execution to complete.

This method blocks until the graph execution (started by run\_async or interrupted by interrupt()) completes. Should be called after interrupt() to ensure the scheduler has fully stopped before performing cleanup operations.

Only call this if interrupt() returned true. Calling wait() when the graph is not running can cause issues with concurrent cleanup.

### fragment \[#fragment]

#### Set the pointer to the fragment of the executor

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::fragment(
    Fragment *fragment
)
```

Set the pointer to the fragment of the executor.

**Parameters**

**`fragment`** `Fragment *`

The pointer to the fragment of the executor.

---

#### Get a pointer to \[Fragment]\(fragment) object

```cpp showLineNumbers={false}
Fragment * holoscan::GPUResidentExecutor::fragment()
```

Get a pointer to [Fragment](fragment) object.

**Returns:** The Pointer to [Fragment](fragment) object.

### owns\_context \[#ownscontext]

```cpp showLineNumbers={false}
bool holoscan::GPUResidentExecutor::owns_context()
```

Get whether the context is owned by the executor.

**Returns:** true if the context is owned by the executor. Otherwise, false.

### context\_uint64 \[#contextuint64]

#### Overload 1

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::context_uint64(
    uint64_t context
)
```

#### Overload 2

```cpp showLineNumbers={false}
uint64_t holoscan::GPUResidentExecutor::context_uint64()
```

### extension\_manager \[#extensionmanager]

```cpp showLineNumbers={false}
virtual std::shared_ptr<ExtensionManager> holoscan::GPUResidentExecutor::extension_manager()
```

Get the extension manager.

**Returns:** The shared pointer of the extension manager.

### exception \[#exception]

#### Set the exception

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::exception(
    const std::exception_ptr &e
)
```

Set the exception.

This method is called by the framework to store the exception that occurred during the execution of the fragment. If the exception is set, this exception is rethrown by the framework after the execution of the fragment.

**Parameters**

**`e`** `const std::exception_ptr &`

The exception to store.

---

#### Get the stored exception

```cpp showLineNumbers={false}
const std::exception_ptr & holoscan::GPUResidentExecutor::exception()
```

Get the stored exception.

This method is called by the framework to get the stored exception that occurred during the execution of the fragment. If the exception is set, this exception is rethrown by the framework after the execution of the fragment.

**Returns:** The reference to the stored exception.

### connect\_ports \[#connectports]

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::connect_ports(
    const std::shared_ptr<Operator> &source_op,
    const std::shared_ptr<Operator> &dest_op,
    const std::string &source_port,
    const std::string &destination_port
)
```

Inspect the port specs of a single source\_port -> destination\_port connection and either allocate a shared device buffer or wire an externally-owned device pointer.

**Parameters**

**`source_op`** `const std::shared_ptr<Operator> &`

The upstream operator (owns the output port).

---

**`dest_op`** `const std::shared_ptr<Operator> &`

The downstream operator (owns the input port).

---

**`source_port`** `const std::string &`

Name of the output port on source\_op.

---

**`destination_port`** `const std::string &`

Name of the input port on dest\_op.

---

### allocate\_io\_device\_buffer \[#allocateiodevicebuffer]

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::allocate_io_device_buffer(
    const std::shared_ptr<Operator> &source_op,
    const std::shared_ptr<Operator> &dest_op,
    const std::string &source_port,
    const std::string &target_port,
    size_t memory_block_size
)
```

### connect\_io\_device\_ptr \[#connectiodeviceptr]

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::connect_io_device_ptr(
    const std::shared_ptr<Operator> &source_op,
    const std::shared_ptr<Operator> &dest_op,
    const std::string &source_port,
    const std::string &target_port,
    void *device_ptr
)
```

### create\_gpu\_resident\_cuda\_graph \[#creategpuresidentcudagraph]

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::create_gpu_resident_cuda_graph()
```

This function creates the full GPU-resident CUDA graph.

It also instantiates the CUDA graph to be ready for launch.

### create\_cuda\_graph\_from\_operators \[#createcudagraphfromoperators]

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::create_cuda_graph_from_operators(
    std::vector<std::shared_ptr<Operator>> &topo_ordered_operators,
    cudaGraph_t &graph,
    cudaStream_t capture_stream
)
```

### verify\_distinct\_operator\_names \[#verifydistinctoperatornames]

```cpp showLineNumbers={false}
bool holoscan::GPUResidentExecutor::verify_distinct_operator_names()
```

This function verifies that the operator names are distinct between the main workload fragment and the data ready handler fragment.

Assumes topologically ordered operators are already created before calling this function.

**Returns:** True if the operator names are distinct, false otherwise.

### set\_unique\_ids \[#setuniqueids]

```cpp showLineNumbers={false}
void holoscan::GPUResidentExecutor::set_unique_ids(
    const std::shared_ptr<Operator> &op
)
```

### add\_receivers \[#addreceivers]

```cpp showLineNumbers={false}
virtual bool holoscan::GPUResidentExecutor::add_receivers(
    const std::shared_ptr<Operator> &op,
    const std::string &receivers_name,
    std::vector<std::string> &new_input_labels,
    std::vector<holoscan::IOSpec *> &iospec_vector
)
```

Add the receivers as input ports of the given operator.

This method is to be called by the Fragment::add\_flow() method to support for the case where the destination input port label points to the parameter name of the downstream operator, and the parameter type is 'std::vector\<holoscan::IOSpec\*>'. This finds a parameter with with 'std::vector\<holoscan::IOSpec\*>' type and create a new input port with a specific label ('parameter name:index'. e.g, 'receivers:0').

**Returns:** true if the receivers are added successfully. Otherwise, false.

**Parameters**

**`op`** `const std::shared_ptr<Operator> &`

The reference to the shared pointer of the operator.

---

**`receivers_name`** `const std::string &`

The name of the receivers whose parameter type is 'std::vector\<holoscan::IOSpec\*>'.

---

**`new_input_labels`** `std::vector<std::string> &`

The reference to the vector of input port labels to which the input port labels are added. In the case of multiple receivers, the input port label is updated to 'parameter name:index' (e.g. 'receivers' => 'receivers:0').

---

**`iospec_vector`** `std::vector<holoscan::IOSpec *> &`

The reference to the vector of [IOSpec](iospec) pointers.

---

### add\_control\_flow \[#addcontrolflow]

```cpp showLineNumbers={false}
virtual bool holoscan::GPUResidentExecutor::add_control_flow(
    const std::shared_ptr<Operator> &upstream_op,
    const std::shared_ptr<Operator> &downstream_op
)
```

Add a control flow between two operators.

This method is called by Fragment::add\_flow() to add a control flow between two operators.

**Returns:** true if the control flow is added successfully. Otherwise, false.

**Parameters**

**`upstream_op`** `const std::shared_ptr<Operator> &`

The shared pointer to the upstream operator.

---

**`downstream_op`** `const std::shared_ptr<Operator> &`

The shared pointer to the downstream operator.

---

---

## Member variables

| Name                                | Type                                                                                        | Description                                                                      |
| ----------------------------------- | ------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
| `fragment_initialized_`             | `bool`                                                                                      |                                                                                  |
| `io_device_buffers_`                | `std::unordered_map< std::string, std::shared_ptr< holoscan::utils::cuda::DeviceBuffer > >` | [Map](map) of input/output port name to the device buffers (executor-allocated). |
| `io_device_ptrs_`                   | `std::unordered_map< std::string, void * >`                                                 | [Map](map) of input/output port name to externally-owned device pointers.        |
| `topo_ordered_main_operators_`      | `std::vector< std::shared_ptr< Operator > >`                                                | [Vector](../typedefs/vector) of topologically ordered operators.                 |
| `topo_ordered_drh_operators_`       | `std::vector< std::shared_ptr< Operator > >`                                                | topologically ordered operators of the data ready handler fragment               |
| `execution_times_us_dev_`           | `std::shared_ptr< holoscan::utils::cuda::DeviceBuffer >`                                    | Device buffer to store the execution times in microseconds.                      |
| `start_time_ns_dev_`                | `std::shared_ptr< holoscan::utils::cuda::DeviceBuffer >`                                    |                                                                                  |
| `actual_samples_collected_dev_`     | `std::shared_ptr< holoscan::utils::cuda::DeviceBuffer >`                                    | Device buffer to store the actual number of samples collected.                   |
| `perf_enabled_`                     | `bool`                                                                                      |                                                                                  |
| `num_samples_`                      | `unsigned int`                                                                              |                                                                                  |
| `sync_with_host_`                   | `bool`                                                                                      |                                                                                  |
| `exec_context_`                     | `std::shared_ptr< ExecutionContext >`                                                       |                                                                                  |
| `timeout_ms_`                       | `unsigned long long`                                                                        |                                                                                  |
| `data_not_ready_sleep_interval_us_` | `unsigned int`                                                                              |                                                                                  |
| `graph_capture_stream_`             | `std::shared_ptr< cudaStream_t >`                                                           |                                                                                  |
| `drh_capture_stream_`               | `std::shared_ptr< cudaStream_t >`                                                           |                                                                                  |
| `drh_graph_`                        | `cudaGraph_t`                                                                               | The CUDA graph of the data ready handler.                                        |
| `workload_graph_`                   | `cudaGraph_t`                                                                               | The CUDA graph of the main workload.                                             |
| `gpu_resident_graph_`               | `cudaGraph_t`                                                                               | The full GPU-resident CUDA graph including control flow nodes.                   |
| `data_ready_handler_fragment_`      | `std::shared_ptr< Fragment >`                                                               |                                                                                  |
| `gpu_resident_deck_`                | `std::shared_ptr< GPUResidentDeck >`                                                        |                                                                                  |
| `fragment_`                         | `Fragment *`                                                                                | The fragment of the executor.                                                    |
| `context_`                          | `void *`                                                                                    | The context.                                                                     |
| `owns_context_`                     | `bool`                                                                                      | Whether the context is owned by the executor.                                    |
| `extension_manager_`                | `std::shared_ptr< ExtensionManager >`                                                       | The extension manager.                                                           |
| `exception_`                        | `std::exception_ptr`                                                                        | The stored exception.                                                            |