Class GPUResidentExecutor

• Defined in File gpu_resident_executor.hpp

Base Type

• public holoscan::Executor (Class Executor)

class GPUResidentExecutor : public holoscan::Executor

Public Functions

GPUResidentExecutor() = delete

inline explicit GPUResidentExecutor(Fragment *fragment)

Construct a new GPUResidentExecutor object.

Parameters

fragment – The pointer to the fragment of the executor.

~GPUResidentExecutor()

virtual void run(OperatorGraph &graph) override

Run the graph.

Parameters

graph – The reference to the graph.

virtual std::future<void> run_async(OperatorGraph &graph) override

Run the graph asynchronously.

Parameters

graph – The reference to the graph.

Returns

The future object.

inline virtual void context(void *context) override

Set the context.

Parameters

context – The context.

virtual bool initialize_fragment() override

Initialize the fragment_ in this Executor.

This method is called by run() to initialize the fragment and the graph of operators in the fragment before execution.

Returns

true if fragment initialization is successful. Otherwise, false.

virtual bool initialize_operator(Operator *op) override

Initialize the given operator.

This method is called by Operator::initialize() to initialize the operator.

Depending on the type of the operator, this method may be overridden to initialize the operator. For example, the default executor (GXFExecutor) initializes the operator using the GXF API and sets the operator’s ID to the ID of the GXF codelet.

Parameters

op – The pointer to the operator.

Returns

true if the operator is initialized successfully. Otherwise, false.

inline virtual bool initialize_scheduler(Scheduler *sch) override

Initialize the given scheduler.

This method is called by Scheduler::initialize() to initialize the operator.

Depending on the type of the scheduler, this method may be overridden to initialize the scheduler. For example, the default executor (GXFExecutor) initializes the scheduler using the GXF API and sets the operator’s ID to the ID of the GXF scheduler.

Parameters

sch – The pointer to the scheduler.

Returns

true if the scheduler is initialized successfully. Otherwise, false.

inline virtual bool initialize_network_context(NetworkContext *network_context) override

Initialize the given network context.

This method is called by NetworkContext::initialize() to initialize the operator.

Depending on the type of the network context, this method may be overridden to initialize the network context. For example, the default executor (GXFExecutor) initializes the network context using the GXF API and sets the operator’s ID to the ID of the GXF network context.

Parameters

network_context – The pointer to the network context.

Returns

true if the network context is initialized successfully. Otherwise, false.

inline virtual bool initialize_fragment_services() override

Initialize the fragment services for the executor.

This method is called during executor initialization to set up any required fragment services.

Depending on the type of executor, this method may be overridden to initialize specific fragment services. For example, the default executor (GXFExecutor) may initialize fragment services using the GXF API.

Returns

true if the fragment services are initialized successfully. Otherwise, false.

void prepare_data_flow(std::shared_ptr<OperatorGraph> graph)

This function prepares the data flow connections between operators It allocates a device memory block for each connection according to the memory block size specified in the operator spec.

Parameters

graph – The operator graph

void initialize_cuda()

This function initializes CUDA. Currently, it sets the device to 0 by default. Setting a different GPU device for GPU-resident execution is not yet supported.

void *device_memory(std::shared_ptr<Operator> op, const std::string &port_name)

This function returns the device memory address of an input or output port corresponding to a given port name. GPU-resident operators use this function to get the device memory address of the input or output port.

Parameters

• op – The operator

• port_name – The name of the input or output port

Returns

The device memory address of the input or output port

virtual bool verify_graph_topology(OperatorGraph &graph)

This function verifies the graph topology is supported by the GPU-resident execution. Currently, it checks if the graph is a linear chain of operators.

Parameters

graph – The operator graph

Returns

True if the graph topology is supported by the GPU-resident execution, false otherwise

void create_gpu_resident_cuda_graph()

This function creates the full GPU-resident CUDA graph. It also instantiates the CUDA graph to be ready for launched.

void timeout_ms(unsigned long long timeout_ms)

void tear_down()

Sends a tear down signal to the GPU-resident CUDA graph.

bool result_ready()

Indicates whether the result of a single iteration of the GPU-resident CUDA graph is ready or not.

Returns

true if the result is ready, false otherwise.

void data_ready()

This function informs GPU-resident CUDA graph that the data is ready for the main workload.

bool is_launched()

Indicates whether the GPU-resident CUDA graph has been launched.

Returns

true if the CUDA graph has been launched, false otherwise.

inline std::shared_ptr<ExecutionContext> execution_context()

Get the execution context - currently, this has no meaning for GPU-resident execution When we need to store something for execution context, we will store a pointer in the exec_context_ for a ExecutionContext object

std::shared_ptr<cudaStream_t> graph_capture_stream()