nemo_rl.weight_sync.refit_actor#

Run the weight synchronizer in its own Ray actor (async_rl.refit_actor).

RefitActor holds the synchronizer in a separate CPU process. RemoteWeightSynchronizer is the controller’s proxy: it forwards each call and keeps worker handles and refit membership in step with the actor’s copy.

Module Contents#

Classes#

RefitActorStatus

What the controller reads back from the RefitActor between refits.

RefitActor

Owns the weight synchronizer; status answers while a refit is stuck.

RemoteWeightSynchronizer

Controller-side WeightSynchronizer that forwards each call to a RefitActor.

Functions#

_noop_shutdown

_disarm_owner

Disarm a generation or policy copy and the cluster its worker group holds.

_disarm_generation

Disarm the RefitActor’s copy of the generation object.

_disarm_synchronizer

Disarm the copies an NCCL synchronizer brings into the RefitActor.

Data#

API#

nemo_rl.weight_sync.refit_actor._CONTROL_CALL_TIMEOUT_S#

120.0

nemo_rl.weight_sync.refit_actor._noop_shutdown() → bool#
nemo_rl.weight_sync.refit_actor._disarm_owner(owner: Any) → None#

Disarm a generation or policy copy and the cluster its worker group holds.

nemo_rl.weight_sync.refit_actor._disarm_generation(generation: Any) → None#

Disarm the RefitActor’s copy of the generation object.

nemo_rl.weight_sync.refit_actor._disarm_synchronizer(
synchronizer: nemo_rl.weight_sync.refit_actor.NcclWeightSynchronizer,
) → None#

Disarm the copies an NCCL synchronizer brings into the RefitActor.

class nemo_rl.weight_sync.refit_actor.RefitActorStatus#

What the controller reads back from the RefitActor between refits.

is_stale: bool#

None

built_membership: Optional[nemo_rl.weight_sync.membership.RefitMembership]#

None

class nemo_rl.weight_sync.refit_actor.RefitActor(
synchronizer: nemo_rl.weight_sync.refit_actor.NcclWeightSynchronizer,
)#

Owns the weight synchronizer; status answers while a refit is stuck.

Initialization

ping() → bool#
update_generation(generation: Any) → None#
init_communicator() → Optional[nemo_rl.weight_sync.membership.RefitMembership]#
sync_weights(
kv_scales: Optional[dict[str, float]],
) → Optional[dict[str, float]]#
reconcile_communicator(
absent_shards: collections.abc.Sequence[int],
force: bool,
) → tuple[Optional[bool], Optional[nemo_rl.weight_sync.membership.RefitMembership]]#
status() → nemo_rl.weight_sync.refit_actor.RefitActorStatus#
shutdown() → None#
class nemo_rl.weight_sync.refit_actor.RemoteWeightSynchronizer(
synchronizer: nemo_rl.weight_sync.interfaces.WeightSynchronizer,
generation: Any,
)#

Bases: nemo_rl.weight_sync.interfaces.WeightSynchronizer

Controller-side WeightSynchronizer that forwards each call to a RefitActor.

Parameters:
  • synchronizer – an initialized NCCL synchronizer (the config check guarantees one), shipped to the actor.

  • generation – the controller’s generation object that synchronizer was built over.

Initialization

attach(generation: Any) → None#

Bind a proxy unpickled in another process to that process’s generation.

raise_if_actor_died(error: BaseException) → None#

Raise RuntimeError if the RefitActor is gone; a worker’s death passes through.

_get(
ref: ray.ObjectRef,
timeout: Optional[float] = _CONTROL_CALL_TIMEOUT_S,
) → Any#
_require_generation() → Any#
_worker_identity() → tuple[int, ...]#
_send_generation_if_changed() → None#
_mirror_membership(
built: Optional[nemo_rl.weight_sync.membership.RefitMembership],
) → None#
submit_sync_weights(
kv_scales: Optional[dict[str, float]] = None,
) → ray.ObjectRef#

Start a refit in the actor and return its ObjectRef, for an async caller.

sync_weights(
*,
timer: Optional[nemo_rl.utils.timer.Timer] = None,
kv_scales: Optional[dict[str, float]] = None,
) → Optional[dict[str, float]]#
property is_stale: bool#
init_communicator() → None#
reconcile_communicator(
absent_shards: collections.abc.Sequence[int],
force: bool = False,
) → Optional[bool]#
shutdown() → None#
__getstate__() → dict[str, Any]#
__setstate__(state: dict[str, Any]) → None#