> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/holoscan/sdk-user-guide/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/holoscan/sdk-user-guide/_mcp/server. # holoscan::CudaStreamHandler > This class handles usage of CUDA streams for operators. This class handles usage of CUDA streams for operators. When using CUDA operations the default stream '0' synchronizes with all other streams in the same context, see [https://docs.nvidia.com/cuda/cuda-runtime-api/stream-sync-behavior.html#stream-sync-behavior](https://docs.nvidia.com/cuda/cuda-runtime-api/stream-sync-behavior.html#stream-sync-behavior). This can reduce performance. The `CudaStreamHandler` class manages streams across operators and makes sure that CUDA operations are properly chained. Usage: * add an instance of `CudaStreamHandler` to your operator * call CudaStreamHandler::register\_interface(spec) from the operator setup() function * in the compute() function call CudaStreamHandler::from\_message(), this will get the CUDA stream from the message of the previous operator. When the operator receives multiple messages, then call CudaStreamHandler::from\_messages(). This will synchronize with multiple streams. * when executing CUDA functions CudaStreamHandler::get() to get the CUDA stream which should be used by your CUDA function * before publishing the output message(s) of your operator call CudaStreamHandler::to\_message() on each message. This will add the CUDA stream used by the CUDA functions in your operator to the output message. ```cpp showLineNumbers={false} #include ``` --- ## Constructors ### Destructor \[#destructor] ### \~CudaStreamHandler ```cpp showLineNumbers={false} holoscan::CudaStreamHandler::~CudaStreamHandler() ``` Destroy the `CudaStreamHandler` object. --- ## Methods ### define\_params \[#defineparams] ```cpp showLineNumbers={false} void holoscan::CudaStreamHandler::define_params( OperatorSpec &spec, bool required = false ) ``` Define the parameters used by this class. **Parameters** **`spec`** `OperatorSpec &` [OperatorSpec](operatorspec) to define the cuda\_stream\_pool parameter --- **`required`** `bool` — default: false If set then it's required that the CUDA stream pool is specified --- ### defineParams \[#defineparams] ```cpp showLineNumbers={false} void holoscan::CudaStreamHandler::defineParams( OperatorSpec &spec, bool required = false ) ``` Define the parameters used by this class. This method is deprecated in favor of [`define_params`](#defineparams). #### Deprecated since 1.0 **Parameters** **`spec`** `OperatorSpec &` [OperatorSpec](operatorspec) to define the cuda\_stream\_pool parameter --- **`required`** `bool` — default: false If set then it's required that the CUDA stream pool is specified --- ### from\_message \[#frommessage] ```cpp showLineNumbers={false} gxf_result_t holoscan::CudaStreamHandler::from_message( gxf_context_t context, const nvidia::gxf::Expected &message ) ``` Get the CUDA stream for the operation from the incoming message. **Returns:** gxf\_result\_t ### fromMessage \[#frommessage] ```cpp showLineNumbers={false} gxf_result_t holoscan::CudaStreamHandler::fromMessage( gxf_context_t context, const nvidia::gxf::Expected &message ) ``` Get the CUDA stream for the operation from the incoming message. This method is deprecated in favor of [`from_message`](#frommessage). #### Deprecated since 1.0 **Returns:** gxf\_result\_t ### from\_messages \[#frommessages] #### Overload 1 ```cpp showLineNumbers={false} gxf_result_t holoscan::CudaStreamHandler::from_messages( gxf_context_t context, const std::vector &messages ) ``` Get the CUDA stream for the operation from the incoming messages ([holoscan::gxf::Entity](../namespaces/gxf/classes/entity) variant). **Returns:** gxf\_result\_t #### Overload 2 ```cpp showLineNumbers={false} gxf_result_t holoscan::CudaStreamHandler::from_messages( gxf_context_t context, const std::vector &messages ) ``` Get the CUDA stream for the operation from the incoming messages (nvidia::gxf::Entity variant). **Returns:** gxf\_result\_t #### Overload 3 ```cpp showLineNumbers={false} gxf_result_t holoscan::CudaStreamHandler::from_messages( gxf_context_t context, size_t message_count, const nvidia::gxf::Entity *messages ) ``` ### fromMessages \[#frommessages] ```cpp showLineNumbers={false} gxf_result_t holoscan::CudaStreamHandler::fromMessages( gxf_context_t context, const std::vector &messages ) ``` Get the CUDA stream for the operation from the incoming messages. This method is deprecated in favor of [`from_messages`](#frommessages). #### Deprecated since 1.0 **Returns:** gxf\_result\_t ### to\_message \[#tomessage] ```cpp showLineNumbers={false} gxf_result_t holoscan::CudaStreamHandler::to_message( nvidia::gxf::Expected &message ) ``` Add the used CUDA stream to the outgoing message. **Returns:** gxf\_result\_t ### toMessage \[#tomessage] ```cpp showLineNumbers={false} gxf_result_t holoscan::CudaStreamHandler::toMessage( nvidia::gxf::Expected &message ) ``` Add the used CUDA stream to the outgoing message. This method is deprecated in favor of [`to_message`](#tomessage). #### Deprecated since 1.0 **Returns:** gxf\_result\_t ### get\_stream\_handle \[#getstreamhandle] ```cpp showLineNumbers={false} nvidia::gxf::Handle holoscan::CudaStreamHandler::get_stream_handle( gxf_context_t context ) ``` Get the CUDA stream handle which should be used for CUDA commands. **Returns:** [nvidia::gxf::Handle\](../../nvidia/namespaces/gxf/classes/handle) ### getStreamHandle \[#getstreamhandle] ```cpp showLineNumbers={false} nvidia::gxf::Handle holoscan::CudaStreamHandler::getStreamHandle( gxf_context_t context ) ``` Get the CUDA stream handle which should be used for CUDA commands. This method is deprecated in favor of [`get_stream_handle`](#getstreamhandle). #### Deprecated since 1.0 **Returns:** [nvidia::gxf::Handle\](../../nvidia/namespaces/gxf/classes/handle) ### get\_cuda\_stream \[#getcudastream] ```cpp showLineNumbers={false} cudaStream_t holoscan::CudaStreamHandler::get_cuda_stream( gxf_context_t context ) ``` Get the CUDA stream which should be used for CUDA commands. If no message stream is set and no stream can be allocated, return the default stream. **Returns:** cudaStream\_t ### getCudaStream \[#getcudastream] ```cpp showLineNumbers={false} cudaStream_t holoscan::CudaStreamHandler::getCudaStream( gxf_context_t context ) ``` Get the CUDA stream which should be used for CUDA commands. If no message stream is set and no stream can be allocated, return the default stream. This method is deprecated in favor of [`get_cuda_stream`](#getcudastream). #### Deprecated since 1.0 **Returns:** cudaStream\_t ### allocate\_internal\_stream \[#allocateinternalstream] ```cpp showLineNumbers={false} gxf_result_t holoscan::CudaStreamHandler::allocate_internal_stream( gxf_context_t context ) ``` Allocate the internal CUDA stream. **Returns:** gxf\_result\_t --- ## Member variables | Name | Type | Description | | ----------------------------- | ------------------------------------------------ | --------------------------------------------------------------------------------------------------------------- | | `cuda_stream_pool_required_` | `bool` | if set then it's required that the CUDA stream pool is specified, if this is not the case an error is generated | | `cuda_stream_pool_` | `Parameter< std::shared_ptr< CudaStreamPool > >` | CUDA stream pool used to allocate the internal CUDA stream. | | `default_stream_warning_` | `bool` | If the CUDA stream pool is not set and we can't use the incoming CUDA stream, issue a warning once. | | `cuda_events_` | `std::vector< cudaEvent_t >` | Array of CUDA events used to synchronize the internal CUDA stream with multiple incoming streams. | | `message_cuda_stream_handle_` | `nvidia::gxf::Handle< nvidia::gxf::CudaStream >` | The CUDA stream which is attached to the incoming message. | | `cuda_stream_handle_` | `nvidia::gxf::Handle< nvidia::gxf::CudaStream >` | Allocated internal CUDA stream handle. |