Execution Resources

Most NVIDIA cuVS C++ APIs accept raft::resources const& as the first argument. Resource objects carry execution state such as CUDA streams, CUDA library handles, memory resources, stream pools, and communication resources.

raft::resources

Source header: raft/core/resources.hpp

Primary execution context passed to most NVIDIA cuVS C++ APIs. The object gives algorithms access to shared CUDA streams, library handles, memory resources, and other lazily-created state.

1
class resources;

raft::resource::get_cuda_stream

Source header: raft/core/resource/cuda_stream.hpp

Returns the CUDA stream associated with a resources object.

1
rmm::cuda_stream_view get_cuda_stream(raft::resources const& res);

Parameters

Returns

rmm::cuda_stream_view

raft::resource::sync_stream

Source header: raft/core/resource/cuda_stream.hpp

Synchronizes the CUDA stream associated with a resources object.

1
void sync_stream(raft::resources const& res);
2
void sync_stream(raft::resources const& res, rmm::cuda_stream_view stream);

Parameters

Returns

void

raft::resource::set_cuda_stream_pool

Source header: raft/core/resource/cuda_stream_pool.hpp

Attaches a CUDA stream pool to a resources object so algorithms can issue independent work on multiple streams.

1
void set_cuda_stream_pool(raft::resources const& res,
2
                          std::shared_ptr<rmm::cuda_stream_pool> pool);

Parameters

Returns

void

raft::resource::get_stream_from_stream_pool

Source header: raft/core/resource/cuda_stream_pool.hpp

Returns a stream from the configured stream pool.

1
rmm::cuda_stream_view get_stream_from_stream_pool(raft::resources const& res);

Parameters

Returns

rmm::cuda_stream_view

raft::resource::sync_stream_pool

Source header: raft/core/resource/cuda_stream_pool.hpp

Synchronizes streams in the configured stream pool.

1
void sync_stream_pool(raft::resources const& res);

Parameters

Returns

void

raft::resource::set_workspace_to_pool_resource

Source header: raft/core/resource/workspace_resource.hpp

Configures workspace allocation for algorithms that need temporary device memory.

1
void set_workspace_to_pool_resource(
2
  raft::resources const& res,
3
  std::optional<std::size_t> allocation_limit = std::nullopt);

Parameters

Returns

void

raft::resource::comms_initialized

Source header: raft/core/resource/comms.hpp

Reports whether communication resources have been initialized.

1
bool comms_initialized(raft::resources const& res);

Parameters

Returns

bool

raft::resource::is_multi_gpu

Source header: raft/core/resource/comms.hpp

Reports whether a resources object is configured for multi-GPU use.

1
bool is_multi_gpu(raft::resources const& res);

Parameters

Returns

bool

raft::device_resources

Source header: raft/core/device_resources.hpp

Convenience raft::resources implementation for single-GPU applications and examples.

1
class device_resources;

raft::device_resources::device_resources

Constructs a single-GPU resources object.

1
device_resources(
2
  rmm::cuda_stream_view stream_view = rmm::cuda_stream_per_thread,
3
  std::shared_ptr<rmm::cuda_stream_pool> stream_pool = nullptr,
4
  std::shared_ptr<rmm::mr::device_memory_resource> workspace_resource = nullptr,
5
  std::optional<std::size_t> allocation_limit = std::nullopt);

Parameters

raft::device_resources::sync_stream

Synchronizes either the main stream or a specific CUDA stream.

1
void sync_stream() const;
2
void sync_stream(rmm::cuda_stream_view stream) const;

Parameters

Returns

void

raft::device_resources::get_stream

Returns the main CUDA stream associated with the resources object.

1
rmm::cuda_stream_view get_stream() const;

Returns

rmm::cuda_stream_view

raft::device_resources::is_stream_pool_initialized

Reports whether a CUDA stream pool is configured.

1
bool is_stream_pool_initialized() const;

Returns

bool

raft::device_resources::get_stream_pool

Returns the configured CUDA stream pool.

1
rmm::cuda_stream_pool const& get_stream_pool() const;

Returns

rmm::cuda_stream_pool const&

raft::device_resources::get_stream_from_stream_pool

Returns a stream from the configured CUDA stream pool.

1
rmm::cuda_stream_view get_stream_from_stream_pool() const;
2
rmm::cuda_stream_view get_stream_from_stream_pool(std::size_t stream_idx) const;

Parameters

Returns

rmm::cuda_stream_view

raft::device_resources::get_next_usable_stream

Returns a stream from the pool when one exists; otherwise returns the main stream.

1
rmm::cuda_stream_view get_next_usable_stream() const;
2
rmm::cuda_stream_view get_next_usable_stream(std::size_t stream_idx) const;

Parameters

Returns

rmm::cuda_stream_view

raft::device_resources::sync_stream_pool

Synchronizes all streams in the pool or a subset of stream indices.

1
void sync_stream_pool() const;
2
void sync_stream_pool(std::vector<std::size_t> stream_indices) const;

Parameters

Returns

void

raft::device_resources::wait_stream_pool_on_stream

Makes the stream pool wait on work submitted to the main stream.

1
void wait_stream_pool_on_stream() const;

Returns

void

raft::device_resources_snmg

Source header: raft/core/device_resources_snmg.hpp

Single-node multi-GPU resources object used by C++ APIs that operate over more than one local GPU.

1
class device_resources_snmg;

raft::device_resources_snmg::device_resources_snmg

Constructs single-node multi-GPU resources for all GPUs or a subset.

1
device_resources_snmg();
2
device_resources_snmg(std::vector<int> const& device_ids);
3
device_resources_snmg(device_resources_snmg const& world);

Parameters

raft::device_resources_snmg::set_memory_pool

Configures a memory pool on all GPUs managed by the resources object.

1
void set_memory_pool(int percent_of_free_memory);

Parameters

Returns

void

raft::device_resources_snmg::has_resource_factory

Reports whether a resource factory is registered for a resource type.

1
bool has_resource_factory(raft::resource::resource_type resource_type) const;

Parameters

Returns

bool