Is this page helpful?

Bootstrap#

enum class rapidsmpf::bootstrap::BackendType#

Backend types for process coordination and bootstrapping.

Values:

enumerator AUTO#

Automatically detect the best backend based on environment.

Detection order:

  1. Socket-based (if RRUN_SOCKET_ADDR set by rrun)

  2. File-based (if RRUN_COORD_DIR set by rrun, backward compatibility)

  3. Slurm/PMIx (if SLURM environment detected)

  4. File-based (default fallback)

enumerator FILE#

File-based coordination using a shared directory.

Uses filesystem for rank coordination and address exchange. Works on single-node and multi-node with shared storage (e.g., NFS) via SSH. Requires RRUN_RANK, RRUN_NRANKS, RRUN_COORD_DIR environment variables.

enumerator SOCKET#

Socket-based coordination using an in-process TCP server (rrun-hosted).

The rrun launcher starts a TCP server on 127.0.0.1 before forking ranks and passes the ephemeral port and a 256-bit random token to all child processes.

Environment variables (set by rrun):

  • RRUN_SOCKET_ADDR: “host:port” of the coordinator server

  • RRUN_SOCKET_TOKEN: 64-hex-char authentication token

  • RRUN_RANK: This process’s rank (0-indexed)

  • RRUN_NRANKS: Total number of ranks

enumerator SLURM#

Slurm-based coordination using PMIx.

Uses PMIx (Process Management Interface for Exascale) for scalable process coordination without requiring a shared filesystem. Designed for Slurm clusters and supports multi-node deployments.

Run with: srun --mpi=pmix -n <nranks> ./program

Environment variables (automatically set by Slurm):

  • PMIX_NAMESPACE: PMIx namespace identifier

  • SLURM_PROCID: Process rank

  • SLURM_NPROCS/SLURM_NTASKS: Total number of processes

using rapidsmpf::bootstrap::Rank = std::int32_t#

Type alias for communicator::Rank.

using rapidsmpf::bootstrap::Duration = std::chrono::duration<double>#

Type alias for Duration type.

constexpr std::size_t rapidsmpf::bootstrap::max_key_size = 255#

Maximum allowed length (in bytes) for a KV coordination key.

Enforced uniformly by all backend implementations:

  • FileBackend: matches the POSIX NAME_MAX filename limit (255 bytes).

  • SocketBackend: protocol field width (%255s) matches this value.

Keys must also be valid as POSIX filename components: no whitespace, path separators (/, \), path traversal sequences (e.g. ..), or null bytes.

Context rapidsmpf::bootstrap::init(
BackendType type = BackendType::AUTO
)#

Initialize the bootstrap context from environment variables.

This function reads environment variables to determine rank, nranks, and backend configuration. It should be called early in the application lifecycle.

Environment variables checked (in order of precedence):

  • RRUN_RANK: Explicitly set rank

  • RRUN_NRANKS: Explicitly set total rank count

  • RRUN_COORD_DIR: File-based coordination directory

auto ctx = rapidsmpf::bootstrap::init();
std::cout << "I am rank " << ctx.rank << " of " << ctx.nranks << std::endl;

Parameters:

type – Backend type to use (default: AUTO for auto-detection).

Throws:

std::runtime_error – if environment is not properly configured.

Returns:

Context object containing rank and coordination information.

void rapidsmpf::bootstrap::barrier(Context const &ctx)#

Perform a barrier synchronization across all ranks.

This ensures all ranks reach this point before any rank proceeds.

Parameters:

ctx – Bootstrap context.

void rapidsmpf::bootstrap::sync(Context const &ctx)#

Ensure all previous put() operations are globally visible.

Different backends have different visibility semantics for put() operations:

  • Slurm/PMIx: Requires explicit fence (PMIx_Fence) to make data visible across nodes.

  • FILE: put() operations are immediately visible via atomic filesystem operations.

This function abstracts these differences. Call sync() after put() operations to ensure data is visible to other ranks before they attempt get().

Parameters:

ctx – Bootstrap context.

void rapidsmpf::bootstrap::put(
Context const &ctx,
std::string const &key,
std::string_view value
)#

Store a key-value pair in the coordination backend (rank 0 only).

Only rank 0 should call this function. The key-value pair is made visible to all ranks after a sync() call. Use this for custom coordination such as UCXX address exchange.

Parameters:

  • ctx – Bootstrap context.

  • key – Key name.

  • value – Value to store.

Throws:

std::runtime_error – if called by non-zero rank.

std::string rapidsmpf::bootstrap::get(
Context const &ctx,
std::string const &key,
Duration timeout = std::chrono::seconds{30}
)#

Retrieve a value from the coordination backend.

Any rank (including rank 0) can call this function to retrieve values published by rank 0. This function blocks until the key is available or timeout occurs.

Parameters:

  • ctx – Bootstrap context.

  • key – Key name to retrieve.

  • timeout – Timeout duration.

Throws:

std::runtime_error – if key not found within timeout.

Returns:

Value associated with the key.

void rapidsmpf::bootstrap::validate_key(std::string const &key)#

Validate a key against all constraints documented in types.hpp.

Enforces the following rules uniformly across all backends:

  • Length must not exceed max_key_size bytes.

  • Must not be empty.

  • Must not contain whitespace, path separators (/, \), path traversal sequences (e.g. ..), or null bytes.

Parameters:

key – Key to validate.

Throws:

std::invalid_argument – if the key violates any constraint.

std::optional<std::string> rapidsmpf::bootstrap::getenv_optional(
std::string_view name
)#

Get environment variable as optional string.

Retrieves the value of an environment variable by name, returning it as std::optional<std::string>. Returns std::nullopt if the variable is not set.

Parameters:

name – Name of the environment variable to retrieve.

Returns:

Value of the environment variable, or std::nullopt if not set.

std::optional<int> rapidsmpf::bootstrap::getenv_int(
std::string_view name
)#

Parse integer from environment variable.

Retrieves an environment variable and parses it as an integer.

Parameters:

name – Name of the environment variable to retrieve.

Throws:

std::runtime_error – if the variable is set but cannot be parsed as an integer.

Returns:

Parsed integer value, or std::nullopt if not set.

std::string rapidsmpf::bootstrap::get_current_cpu_affinity()#

Get current CPU affinity as a string.

Queries the current process’s CPU affinity mask and formats it as a comma-separated list of CPU core IDs, with ranges represented as “start-end”.

Example output: “0-3,8-11” for cores 0,1,2,3,8,9,10,11

Returns:

CPU affinity string, or empty string on error.

std::string rapidsmpf::bootstrap::get_ucx_net_devices()#

Get UCX_NET_DEVICES from environment.

Retrieves the value of the UCX_NET_DEVICES environment variable, which specifies which network devices UCX should use for communication.

Returns:

Value of UCX_NET_DEVICES, or empty string if not set.

int rapidsmpf::bootstrap::get_gpu_id()#

Get GPU ID from CUDA_VISIBLE_DEVICES.

Attempts to determine the GPU ID assigned to this process by checking the CUDA_VISIBLE_DEVICES environment variable.

Returns:

GPU ID (>= 0) if found, -1 otherwise.

bool rapidsmpf::bootstrap::is_running_with_rrun()#

Check if the current process was launched via rrun.

This helper detects bootstrap mode by checking for the presence of the RRUN_RANK environment variable, which is set by rrun.

Returns:

true if running under rrun bootstrap mode, false otherwise.

bool rapidsmpf::bootstrap::is_running_with_slurm()#

Check if the current process is running under Slurm with PMIx.

This helper detects Slurm environment by checking for PMIx namespace or Slurm job step environment variables.

Returns:

true if running under Slurm with PMIx, false otherwise.

Rank rapidsmpf::bootstrap::get_rank()#

Get the current bootstrap rank.

This helper retrieves the rank of the current process when running with a bootstrap launcher (rrun or Slurm). Checks environment variables in order:

  1. RRUN_RANK (set by rrun)

  2. PMIX_RANK (set by PMIx)

  3. SLURM_PROCID (set by Slurm)

Throws:

std::runtime_error – if not running with a bootstrap launcher or if the environment variable cannot be parsed.

Returns:

Rank of the current process.

Rank rapidsmpf::bootstrap::get_nranks()#

Get the number of bootstrap ranks.

This helper retrieves the number of ranks when running with a bootstrap launcher (rrun or Slurm). Checks environment variables in order:

  1. RRUN_NRANKS (set by rrun)

  2. SLURM_NPROCS (set by Slurm)

  3. SLURM_NTASKS (set by Slurm)

Throws:

std::runtime_error – if not running with a bootstrap launcher or if the environment variable cannot be parsed.

Returns:

Number of ranks.

std::vector<int> rapidsmpf::bootstrap::parse_cpu_list(
std::string const &cpulist
)#

Parse CPU list string into vector of core IDs.

Parses a comma-separated CPU list string that may contain ranges (e.g., “0-3,8-11”) into a vector of individual CPU core IDs.

Parameters:

cpulist – CPU list string (e.g., “0-3,8-11” or “0,1,2,3”).

Returns:

Vector of CPU core IDs. Empty if parsing fails or input is empty.

bool rapidsmpf::bootstrap::compare_cpu_affinity(
std::string const &actual,
std::string const &expected
)#

Compare two CPU affinity strings (order-independent).

Compares two CPU affinity strings by parsing them into sorted lists of core IDs and checking if they contain the same cores, regardless of order or formatting.

Parameters:

  • actual – Actual CPU affinity string.

  • expected – Expected CPU affinity string.

Returns:

true if both strings represent the same set of CPU cores, false otherwise.

bool rapidsmpf::bootstrap::compare_device_lists(
std::string const &actual,
std::string const &expected
)#

Compare two comma-separated device lists (order-independent).

Compares two comma-separated device lists by parsing them into sorted vectors and checking if they contain the same devices, regardless of order.

Parameters:

  • actual – Actual device list string.

  • expected – Expected device list string.

Returns:

true if both strings represent the same set of devices, false otherwise.

struct Context#
#include <bootstrap.hpp>

Context information for the current process/rank.

This structure contains the rank assignment and total rank count, along with additional metadata about the execution environment.

Public Members

Rank rank#

This process’s rank (0-indexed).

Rank nranks#

Total number of ranks in the job.

BackendType type#

Backend type used for coordination.

std::optional<std::string> coord_dir#

Coordination directory (for FILE backend).

std::shared_ptr<detail::Backend> backend#

Backend implementation (internal, do not access directly).