Class CudaObjectHandler

• Defined in File cuda_object_handler.hpp

Derived Type

• public holoscan::gxf::CudaObjectHandler (Class CudaObjectHandler)

class CudaObjectHandler

Pure virtual base class for handling CUDA streams in operators.

This class defines the interface for managing CUDA streams in Holoscan operators. It provides methods for stream synchronization, allocation, and management.

Subclassed by holoscan::gxf::CudaObjectHandler

Public Functions

virtual ~CudaObjectHandler() = default

virtual void init_from_operator(Operator *op) = 0

Initialize the handler from an operator.

Parameters

op – The operator this instance is attached to.

virtual int add_stream(const cudaStream_t stream, const std::string &output_port_name) = 0

Add a CUDA stream to an output port.

Parameters

• stream – The stream to add

• output_port_name – The name of the output port

Returns

0 if successful, otherwise an error code

virtual cudaStream_t get_cuda_stream(void *context, const std::string &input_port_name, bool allocate = false, bool sync_to_default = true) = 0

Get the CUDA stream for a given input port.

Parameters

• context – context object (e.g. gxf_context_t for the GXF backend)

• input_port_name – The name of the input port

• allocate – If true, allocate a new stream if none exists

• sync_to_default – If true, synchronize to default stream

Returns

The CUDA stream to use

virtual std::vector<std::optional<cudaStream_t>> get_cuda_streams(void *context, const std::string &input_port_name) = 0

Get all CUDA streams for a given input port.

Parameters

• context – context object (e.g. gxf_context_t for the GXF backend)

• input_port_name – The name of the input port

Returns

Vector of optional CUDA streams (one per message)

virtual int synchronize_streams(std::vector<cudaStream_t> cuda_streams, cudaStream_t target_stream, bool sync_to_default_stream = true) = 0

Synchronize streams with a target stream.

Parameters

• cuda_streams – The streams to synchronize

• target_stream – The stream to synchronize to

• sync_to_default_stream – If true, also sync to default stream

Returns

0 if successful, error code otherwise

virtual int release_internal_streams(void *context) = 0

Release all internally allocated CUDA streams.

Parameters

context – context object (e.g. gxf_context_t for the GXF backend)

Returns

0 if successful, error code otherwise

virtual void clear_received_streams() = 0

Clear all received streams.

This is used to refresh the state before each compute call.