CUDA Streams

class cuda_stream

#include <cuda_stream.hpp>

Owning wrapper for a CUDA stream.

Provides RAII lifetime semantics for a CUDA stream.

Public Types

enum class flags : unsigned int

stream creation flags.

Values:

enumerator sync_default

Created stream synchronizes with the default stream.

enumerator non_blocking

Created stream does not synchronize with the default stream.

Public Functions

cuda_stream(cuda_stream&&) = default

Move constructor (default).

A moved-from cuda_stream is invalid and it is Undefined Behavior to call methods that access the owned stream.

cuda_stream &operator=(cuda_stream&&) = default

Move copy assignment operator (default).

A moved-from cuda_stream is invalid and it is Undefined Behavior to call methods that access the owned stream.

Returns:

A reference to this cuda_stream

cuda_stream(

cuda_stream::flags flags = cuda_stream::flags::sync_default

)

Construct a new CUDA stream object.

Parameters:

flags – Stream creation flags.

Throws:

rmm::cuda_error – if stream creation fails

bool is_valid() const

Returns true if the owned stream is non-null.

Returns:

true If the owned stream has not been explicitly moved and is therefore non-null.

Returns:

false If the owned stream has been explicitly moved and is therefore null.

cudaStream_t value() const

Get the value of the wrapped CUDA stream.

Returns:

cudaStream_t The wrapped CUDA stream.

explicit operator cudaStream_t() const noexcept

Explicit conversion to cudaStream_t.

operator cuda::stream_ref() const noexcept

Implicit conversion to cuda::stream_ref.

Returns:

A stream_ref for the owned stream

void synchronize() const

Synchronize the owned CUDA stream.

Calls cudaStreamSynchronize().

Throws:

rmm::cuda_error – if stream synchronization fails

void synchronize_no_throw() const noexcept

Synchronize the owned CUDA stream. Does not throw if there is an error.

Calls cudaStreamSynchronize() and asserts if there is an error.

class cuda_stream_pool

#include <cuda_stream_pool.hpp>

A pool of CUDA streams.

Provides efficient access to collection of CUDA stream objects.

Successive calls may return a cuda::stream_ref of identical streams. For example, a possible implementation is to maintain a circular buffer of cuda_stream objects.

Public Functions

explicit cuda_stream_pool(

std::size_t pool_size = default_size,

cuda_stream::flags flags = cuda_stream::flags::sync_default

)

Construct a new CUDA stream pool object of the given non-zero size.

Throws:

logic_error – if pool_size is zero

Parameters:

• pool_size – The number of streams in the pool

• flags – Flags used when creating streams in the pool.

cuda::stream_ref get_stream() const noexcept

Get a cuda::stream_ref of a stream in the pool.

This function is thread safe with respect to other calls to the same function.

Returns:

cuda::stream_ref

cuda::stream_ref get_stream(std::size_t stream_id) const

Get a cuda::stream_ref of the stream associated with stream_id. Equivalent values of stream_id return a stream_ref to the same underlying stream.

This function is thread safe with respect to other calls to the same function.

Note

stream_id is wrapped around the pool size, therefore any size_t value is allowed.

Parameters:

stream_id – Unique identifier for the desired stream

Returns:

cuda::stream_ref

std::size_t get_pool_size() const noexcept

Get the number of streams in the pool.

This function is thread safe with respect to other calls to the same function.

Returns:

the number of streams in the pool

Public Static Attributes

static constexpr std::size_t default_size = {16}

Default stream pool size.