Is this page helpful?

Configuration Options#

RapidsMPF can be configured using a dictionary of options, which may be populated via environment variables. All dictionary keys are automatically converted to lowercase. See the Python API Reference for full details.

Each configuration option includes:

  • Name: The key used in the configuration dictionary.

  • Environment Variable: The corresponding environment variable name.

  • Description: Describes what the option controls, including accepted values.

Environment variable names are always uppercase and prefixed with RAPIDSMPF_.

Typically, it is up to the user to read environment variables using code such as:

options = Options()
options.insert_if_absent(get_environment_variables())

It is always explicit in C++, use something like:

rapidsmpf::config::Options options{rapidsmpf::config::get_environment_variables()};

Available Options#

General#

  • log

    • Environment Variable: RAPIDSMPF_LOG

    • Default: WARN

    • Description: Controls the logging verbosity level. Valid values are:

      • NONE: Disable all logging.

      • PRINT: General print messages.

      • WARN: Warning messages (default).

      • INFO: Informational messages.

      • DEBUG: Debug-level messages.

      • TRACE: Fine-grained trace-level messages.

  • statistics

    • Environment Variable: RAPIDSMPF_STATISTICS

    • Default: False

    • Description: Enable RapidsMPF statistics collection.

  • num_streaming_threads

    • Environment Variable: RAPIDSMPF_NUM_STREAMING_THREADS

    • Default: 1

    • Description: Number of threads used to execute coroutines. Must be greater than zero.

  • num_streams

    • Environment Variable: RAPIDSMPF_NUM_STREAMS

    • Default: 16

    • Description: Number of CUDA streams used by RapidsMPF. A pool of CUDA streams is created at startup, and work is scheduled onto these streams to enable concurrent GPU execution and overlap of computation and data movement. Must be greater than zero.

  • memory_reserve_timeout

    • Environment Variable: RAPIDSMPF_MEMORY_RESERVE_TIMEOUT

    • Default: 100 ms

    • Description: Controls the global progress timeout for memory reservation requests. If the value does not include a unit, it is interpreted as seconds.

      The value limits how long the system may go without making progress on any pending memory reservation. When the timeout expires and no reservation has been satisfied, the system forces progress by selecting a pending request and attempting to reserve memory for it. Depending on the context, this may result in an empty reservation, an overbooked reservation, or a failure.

      This option ensures forward progress under memory pressure and prevents the system from stalling indefinitely when memory availability fluctuates.

      A spill frees memory in whole buffers and can take longer than this timeout. So when the timeout expires while a spill is executing, the request keeps waiting for as long as spilling keeps increasing the memory available, then takes one more admission attempt before forcing progress. This extra wait is capped at ten times this timeout, which only matters if a spill never finishes. Nothing is spilled on behalf of the request, so a workload that does not spill sees the timeout unchanged.

      The shuffler uses the same timeout for receive buffers. When no incoming payload could get a buffer for this long, the shuffler allocates one in host memory, even beyond the host memory limit, if allow_overbooking_by_default allows it, and fails with an error otherwise. Only one payload is forced per timeout, so under sustained memory pressure N waiting payloads take about N timeouts to get their buffers. Receive buffers only have to wait when host memory is limited too (see spill_host_limit), since otherwise they always fit in host memory.

  • allow_overbooking_by_default

    • Environment Variable: RAPIDSMPF_ALLOW_OVERBOOKING_BY_DEFAULT

    • Default: true

    • Description: Controls the default overbooking behavior for high-level memory reservation APIs, such as reserve_memory(), and for receive buffers in the shuffler.

      When enabled, high-level memory reservation requests and shuffler receives may overbook memory after the global memory_reserve_timeout expires, allowing forward progress under memory pressure.

      When disabled, they fail with an error if no progress is possible within the timeout.

      This option is only used when a high-level API does not explicitly specify an overbooking policy. It does not change the behavior of lower-level memory reservation primitives or imply that overbooking is enabled or disabled globally across the system.

  • pinned_memory

    • Environment Variable: RAPIDSMPF_PINNED_MEMORY

    • Default: false

    • Description: Enables pinned host memory if it is available on the system. Pinned host memory provides higher bandwidth and lower latency for device-to-host transfers compared to regular pageable host memory. When enabled, RapidsMPF primarily uses pinned host memory for spilling. Availability of pinned host memory can be checked using is_pinned_memory_resources_supported().

  • pinned_initial_pool_size

    • Environment Variable: RAPIDSMPF_PINNED_INITIAL_POOL_SIZE

    • Default: 10% of per-GPU host memory

    • Description: Initial size of the pinned host memory pool when pinned_memory is enabled. When unset or empty, the pool is pre-allocated to 10% of total host memory available in the current NUMA node divided by the number of GPUs in that NUMA node. Accepts byte counts or percentage (e.g. "1GiB", "512MiB").

  • pinned_max_pool_size

    • Environment Variable: RAPIDSMPF_PINNED_MAX_POOL_SIZE

    • Default: 80% of per-GPU host memory

    • Description: Maximum size of the pinned host memory pool when pinned_memory is enabled. When unset or empty, the pool is capped at 80% of total host memory available in the current NUMA node divided by the number of GPUs in that NUMA node. Accepts positive byte counts or percentages (e.g. "4GiB", "2048MiB"). Use "disabled" for an unbounded pool; zero is not valid. The value cannot exceed the host memory of the current NUMA node. See also spill_host_limit, which limits pageable host memory separately and shares the same percentage base.

  • spill_device_limit

    • Environment Variable: RAPIDSMPF_SPILL_DEVICE_LIMIT

    • Default: 80%

    • Description: Soft upper limit on device memory usage that RapidsMPF attempts to stay under by triggering spilling. This limit is a best-effort target and may not always be enforceable. The value can be specified either as an absolute byte count (e.g. "10GiB", "512MB") or as a percentage of the total memory of the current device (e.g. "80%").

  • spill_host_limit

    • Environment Variable: RAPIDSMPF_SPILL_HOST_LIMIT

    • Default: disabled (unbounded)

    • Description: Soft upper limit on pageable host memory used by RapidsMPF for spilling. Accepts byte counts (e.g. "10GiB", "512MB") or a percentage of the host memory available per GPU, the same base as pinned_max_pool_size (e.g. "40%"). Use "disabled" for an unbounded limit; zero is not valid. This limit is what makes host data spill to disk: once it is exceeded, host data is spilled to disk (if disk_spill_dir is set). With the default unbounded limit, host data is never spilled to disk. This limit is independent of pinned_max_pool_size and is not validated against it. Keep the two at or below 80% combined (e.g. pinned_max_pool_size=40% and spill_host_limit=40%), leaving 20% of the host memory available per GPU for the OS and any other untracked host allocations. Since pinned_max_pool_size defaults to 80%, lower it when setting this limit with pinned_memory enabled.

  • periodic_spill_check

    • Environment Variable: RAPIDSMPF_PERIODIC_SPILL_CHECK

    • Default: 1ms

    • Description: Enable periodic spill checks. A dedicated thread continuously checks and performs spilling based on the current available memory as reported by the buffer resource, making headroom in every addressable memory type (device, pinned host and host). The value of periodic_spill_check specifies the pause between checks and supports time units, e.g. us or ms. If no unit is specified, seconds are assumed. Use "disabled" to disable periodic spill checks.

  • disk_spill_dir

    • Environment Variable: RAPIDSMPF_DISK_SPILL_DIR

    • Default: false

    • Description: Directory used for disk spill files. When set to a path, RapidsMPF creates an exclusively owned subdirectory and enables disk I/O. Disabled values (false, none, off, …) leave disk spilling disabled. A whitespace-only value is invalid.

      Disk is the last resort for spilling. Device data reaches disk only when pinned and host memory are both full, and host data reaches disk only once spill_host_limit is exceeded. With the default unbounded spill_host_limit, nothing is ever spilled to disk, so set spill_host_limit as well when you expect disk spilling.

On this page