nemo_rl.data_plane.adapters.transfer_queue_env

Mooncake engine environment, configured before the engine is imported.

Mooncake snapshots its whole MC_* configuration as its extension loads, so these variables only take effect if they are in os.environ beforehand. That makes import order load-bearing, and the failure is silent: a late write lands in os.environ — where it still reads back correctly — while the engine keeps the value it captured. On a rail-isolated RoCE fabric that silence costs a run, with every transfer dying as “transport retry counter exceeded”.

This module therefore deliberately imports nothing from transfer_queue or mooncake, so importing it can never be the thing that loads the engine. Keep it that way — a convenience import here, or one added to this package’s __init__, would defeat the whole point. :func:configure_engine_env turns the ordering violation into an error rather than a silent misconfiguration.

Module Contents

Functions

_engine_already_imported	Return the first engine module already in sys.modules, else None.
rail_link_layers	Map each mlx5 rail to its port-1 link layer, read from sysfs.
fabric_is_roce_only	True when the host has RoCE rails and no InfiniBand.
_wanted_engine_env	The MC_* values this backend needs, for this host’s fabric.
configure_engine_env	Set the mooncake knobs that must be identical in every process.

Data

_ENGINE_MODULES

API

nemo_rl.data_plane.adapters.transfer_queue_env._ENGINE_MODULES

(‘transfer_queue’, ‘mooncake’)

nemo_rl.data_plane.adapters.transfer_queue_env._engine_already_imported() → str | None

Return the first engine module already in sys.modules, else None.

nemo_rl.data_plane.adapters.transfer_queue_env.rail_link_layers() → dict[str, str]

Map each mlx5 rail to its port-1 link layer, read from sysfs.

nemo_rl.data_plane.adapters.transfer_queue_env.fabric_is_roce_only() → bool

True when the host has RoCE rails and no InfiniBand.

Deliberately requires seeing InfiniBand to answer False, so an empty or unreadable sysfs cannot silently opt a RoCE host out of the pairing hint.

nemo_rl.data_plane.adapters.transfer_queue_env._wanted_engine_env() → dict[str, str]

The MC_* values this backend needs, for this host’s fabric.

nemo_rl.data_plane.adapters.transfer_queue_env.configure_engine_env(

cfg: nemo_rl.data_plane.interfaces.DataPlaneConfig,

) → None

Set the mooncake knobs that must be identical in every process.

No-op unless the backend is mooncake_cpu; simple has no engine. Values already present in the environment are left alone, so a launcher can override any of them — except MC_ENABLE_DEST_DEVICE_AFFINITY, which mooncake reads presence-only, so a launcher trying to override it to "0" enables it instead; unsetting it is the only way to disable it.

Call this before anything imports transfer_queue or mooncake.

Func:

nemo_rl.data_plane.factory.maybe_configure_data_plane_env does, on the driver before init_ray, and Ray hands the result to every worker.

Raises:

RuntimeError – if a variable still needs setting but the engine is already imported, i.e. the value can no longer reach it. Fatal on purpose — the alternative is a run that looks configured and is not.