Class CudaStreamPool

• Defined in File cuda_stream_pool.hpp

Base Type

• public holoscan::Allocator (Class Allocator)

class CudaStreamPool : public holoscan::Allocator

CUDA stream pool allocator.

An allocator that creates a pool of CUDA streams.

Internally, the streams created correspond to nvidia::gxf::CudaStream objects whose lifetime is managed by the underlying GXF framework.

The user can pass this resource to any make_operator to allow an operator to control the CUDA stream pool it will use when a stream-related API calls such as InputContext::receive_cuda_stream is made from the Operator::compute method.

==Parameters==

• dev_id (int32_t, optional): The CUDA device id specifying which device the memory pool will use (Default: 0).

• stream_flags (uint32_t, optional): The flags passed to the underlying cudaStreamCreateWithPriority CUDA runtime API call when the streams for this pool are created (Default: 0).

• stream_priority_ (int32_t, optional): The priority passed to the underlying cudaStreamCreateWithPriority CUDA runtime API call when the streams for this pool are created (Default: 0).

• reserved_size (uint32_t, optional): The number of streams to initialize the pool with (Default: 1).

• max_size (uint32_t, optional): The maximum number of streams that can be allocated in the pool. A value of 0 indicates that the size is unlimited (Default: 0). Note that in practice the device hardware will limit the number of possible concurrent kernels and/or memory copy operations to a value defined by CUDA_DEVICE_MAX_CONNECTIONS.

• cuda_green_context (std::shared_ptr, optional): The CUDA green context to create the CUDA stream pool.

• nvtx_identifier (std::string, optional): The NVTX identifier of the stream pool. This identifier will be used in NSight captures to identify the stream pool.

Public Functions

template<typename ArgT, typename ...ArgsT, typename = std::enable_if_t<!std::is_base_of_v<::holoscan::Resource, std::decay_t<ArgT>> && (std::is_same_v<::holoscan::Arg, std::decay_t<ArgT>> || std::is_same_v<::holoscan::ArgList, std::decay_t<ArgT>>)>>
inline explicit CudaStreamPool(ArgT &&arg, ArgsT&&... args)

CudaStreamPool() = default

inline CudaStreamPool(int32_t dev_id, uint32_t stream_flags, int32_t stream_priority, uint32_t reserved_size, uint32_t max_size, std::shared_ptr<CudaGreenContext> cuda_green_context = nullptr, const std::string &nvtx_identifier = "defaultStreamPool")

CudaStreamPool(const std::string &name, nvidia::gxf::CudaStreamPool *component)

inline virtual const char *gxf_typename() const override

virtual void setup(ComponentSpec &spec) override

Define the resource specification.

Parameters

spec – The reference to the component specification.

virtual void initialize() override

Initialize the component.

This method is called only once when the component is created for the first time, and use of light-weight initialization.

nvidia::gxf::CudaStreamPool *get() const

inline int32_t get_dev_id() const