Class CudaStreamPool

• Defined in File cuda_stream_pool.hpp

Base Type

• public holoscan::Allocator (Class Allocator)

class CudaStreamPool : public holoscan::Allocator

CUDA stream pool allocator.

An allocator that creates a pool of CUDA streams.

Internally, the streams created correspond to nvidia::gxf::CudaStream objects whose lifetime is managed by the underlying GXF framework.

The user can pass this resource to any make_operator to allow an operator to control the CUDA stream pool it will use when a stream-related API calls such as InputContext::receive_cuda_stream is made from the Operator::compute method.

==Parameters==

• dev_id (int32_t, optional): The CUDA device id specifying which device the memory pool will use (Default: 0).

• stream_flags (uint32_t, optional): The flags passed to the underlying cudaStreamCreateWithPriority CUDA runtime API call when the streams for this pool are created (Default: 0).

• stream_priority_ (int32_t, optional): The priority passed to the underlying cudaStreamCreateWithPriority CUDA runtime API call when the streams for this pool are created (Default: 0).

• reserved_size (uint32_t, optional): The number of streams to initialize the pool with (Default: 1).

• max_size (uint32_t, optional): The maximum number of streams that can be allocated in the pool. A value of 0 indicates that the size is unlimited (Default: 0). Note that in practice the device hardware will limit the number of possible concurrent kernels and/or memory copy operations to a value defined by CUDA_DEVICE_MAX_CONNECTIONS.

Public Functions

template<typename ArgT, typename ...ArgsT, typename = std::enable_if_t<!std::is_base_of_v<::holoscan::Resource, std::decay_t<ArgT>> && (std::is_same_v<::holoscan::Arg, std::decay_t<ArgT>> || std::is_same_v<::holoscan::ArgList, std::decay_t<ArgT>>)>>
inline explicit CudaStreamPool(ArgT &&arg, ArgsT&&... args)

CudaStreamPool() = default

inline CudaStreamPool(int32_t dev_id, uint32_t stream_flags, int32_t stream_priority, uint32_t reserved_size, uint32_t max_size)

CudaStreamPool(const std::string &name, nvidia::gxf::CudaStreamPool *component)

inline virtual const char *gxf_typename() const override

virtual void setup(ComponentSpec &spec) override

Define the resource specification.

Parameters

spec – The reference to the component specification.

nvidia::gxf::CudaStreamPool *get() const