C++

RapidsMPF exposes a full C++ API for building high-performance distributed GPU workloads without a Python runtime. The C++ layer is the foundation on which the Python bindings are built.

API Reference

• Core API
  • AllowOverbooking
  • MemoryType
  • TrimZeroFraction
  • Rank
  • OpID
  • StageID
  • any_device_resource
  • any_host_device_resource
  • any_host_resource
  • Clock
  • Duration
  • TimePoint
  • COMM_HAVE_UCXX
  • COMM_HAVE_MPI
  • MEMORY_TYPES
  • MEMORY_TYPE_NAMES
  • SPILL_TARGET_MEMORY_TYPES
  • PinnedMemoryDisabled
  • operator<<()
  • cuda_stream_join()
  • cuda_stream_join()
  • buffer_copy()
  • device_limit_from_options()
  • periodic_spill_check_from_options()
  • stream_pool_from_options()
  • cuda_memcpy_batch_async()
  • cuda_memcpy_async()
  • leq_memory_types()
  • to_string()
  • operator<<()
  • operator>>()
  • is_pinned_memory_resources_supported()
  • pinned_pool_properties_from_options()
  • is_host_accessible()
  • is_device_accessible()
  • spill_partitions()
  • unspill_partitions()
  • get_total_host_memory()
  • get_current_numa_node()
  • get_current_numa_nodes()
  • get_numa_node_host_memory()
  • get_host_memory_per_gpu()
  • extract_item()
  • extract_item()
  • extract_value()
  • extract_value()
  • extract_key()
  • extract_key()
  • to_vector()
  • is_running_under_valgrind()
  • safe_div()
  • ceil_div()
  • chunk_indices()
  • contains()
  • owner_equal()
  • safe_cast()
  • trim()
  • to_lower()
  • to_upper()
  • format_nbytes()
  • format_duration()
  • parse_nbytes()
  • parse_nbytes_unsigned()
  • parse_nbytes_or_percent()
  • parse_duration()
  • parse_string()
  • parse_string()
  • parse_optional()
  • parse_string_list()
  • rapidsmpf::Tag
  • rapidsmpf::Communicator
  • rapidsmpf::Logger
  • rapidsmpf::MPI
  • rapidsmpf::Single
  • rapidsmpf::CudaEvent
  • rapidsmpf::cuda_error
  • rapidsmpf::bad_alloc
  • rapidsmpf::out_of_memory
  • rapidsmpf::reservation_error
  • rapidsmpf::BackRefMixin
  • rapidsmpf::Buffer
  • rapidsmpf::StreamPool
  • rapidsmpf::BufferResource
  • rapidsmpf::ContentDescription
  • rapidsmpf::HostBuffer
  • rapidsmpf::HostMemoryResource
  • rapidsmpf::MemoryReservation
  • rapidsmpf::PackedData
  • rapidsmpf::PinnedPoolProperties
  • rapidsmpf::PinnedMemoryResource
  • rapidsmpf::ScopedMemoryRecord
  • rapidsmpf::SpillManager
  • rapidsmpf::OwningWrapper
  • rapidsmpf::ProgressThread
  • rapidsmpf::RmmResourceAdaptor
  • rapidsmpf::Statistics
  • rapidsmpf::StreamOrderedTiming
  • rapidsmpf::overloaded
  • MPI Utilities
• Bootstrap
  • BackendType
  • Rank
  • Duration
  • max_key_size
  • init()
  • barrier()
  • sync()
  • put()
  • get()
  • validate_key()
  • getenv_optional()
  • getenv_int()
  • get_current_cpu_affinity()
  • get_ucx_net_devices()
  • get_gpu_id()
  • is_running_with_rrun()
  • is_running_with_slurm()
  • get_rank()
  • get_nranks()
  • parse_cpu_list()
  • compare_cpu_affinity()
  • compare_device_lists()
  • rapidsmpf::bootstrap::Context
• Collectives
  • ReduceOperator
  • rapidsmpf::coll::AllGather
  • rapidsmpf::coll::AllReduce
  • rapidsmpf::coll::SparseAlltoall
• Configuration
  • OptionFactory
  • DEFAULTS
  • get_environment_variables()
  • get_environment_variables()
  • rapidsmpf::config::OptionValue
  • rapidsmpf::config::Options
• Metadata and Payload Exchange
  • rapidsmpf::communicator::MetadataPayloadExchange
  • rapidsmpf::communicator::TagMetadataPayloadExchange
• Shuffler
  • PartID
  • operator<<()
  • operator<<()
  • rapidsmpf::shuffler::Shuffler
• Streaming Engine
  • Actor
  • Semaphore
  • get_content_description()
  • to_message()
  • get_content_description()
  • get_content_description()
  • to_message()
  • to_message()
  • run_actor_network()
  • coro_results()
  • coro_results()
  • reserve_memory()
  • rapidsmpf::streaming::PartitionMapChunk
  • rapidsmpf::streaming::PartitionVectorChunk
  • rapidsmpf::streaming::AllGather
  • rapidsmpf::streaming::AllReduce
  • rapidsmpf::streaming::ShufflerAsync
  • rapidsmpf::streaming::SparseAlltoall
  • rapidsmpf::streaming::Channel
  • rapidsmpf::streaming::ThrottlingAdaptor
  • rapidsmpf::streaming::ShutdownAtExit
  • rapidsmpf::streaming::Context
  • rapidsmpf::streaming::CoroThreadPoolExecutor
  • rapidsmpf::streaming::Lineariser
  • rapidsmpf::streaming::MemoryReserveOrWait
  • rapidsmpf::streaming::Message
  • rapidsmpf::streaming::BoundedQueue
  • rapidsmpf::streaming::SpillableMessages
  • Actors
• rrun
  • check_binding()
  • get_expected_binding()
  • validate_binding()
  • bind()
  • bind()
  • rapidsmpf::rrun::bind_options
  • rapidsmpf::rrun::resource_binding
  • rapidsmpf::rrun::expected_binding
  • rapidsmpf::rrun::binding_validation
  • rapidsmpf::rrun::ScopedEnvVar

Coverage

The C++ API provides access to all core RapidsMPF subsystems:

• Communicator — MPI and UCXX backends for inter-process communication.

• Shuffler — Out-of-core, distributed payload shuffle service.

• Streaming Engine — Asynchronous multi-GPU pipeline with Channels, Actors, and Messages.

• Memory — BufferResource, spilling, pinned memory, and packed data utilities.

• Config — Configuration options and environment-variable parsing.

Shuffle Service

See Shuffle Architecture for an in-depth explanation of the shuffle design.

rrun — Distributed Launcher

RapidsMPF includes rrun, a lightweight launcher that eliminates the MPI dependency for multi-GPU workloads. See Streaming execution for more on the programming model.

Build rrun

cd cpp/build
cmake --build . --target rrun

Single-Node Launch

# Launch 2 ranks on the local node
./tools/rrun -n 2 ./benchmarks/bench_comm -C ucxx -O all-to-all

# With verbose output and specific GPUs
./tools/rrun -v -n 4 -g 0,1,2,3 ./benchmarks/bench_comm -C ucxx