nemo_rl.weight_sync.refit_actor#
Run the weight synchronizer in its own Ray actor (async_rl.refit_actor).
RefitActor holds the synchronizer in a separate CPU process.
RemoteWeightSynchronizer is the controller’s proxy: it forwards each call and keeps
worker handles and refit membership in step with the actor’s copy.
Module Contents#
Classes#
What the controller reads back from the RefitActor between refits. |
|
Owns the weight synchronizer; |
|
Controller-side |
Functions#
Disarm a generation or policy copy and the cluster its worker group holds. |
|
Disarm the RefitActor’s copy of the generation object. |
|
Disarm the copies an NCCL synchronizer brings into the RefitActor. |
Data#
API#
- nemo_rl.weight_sync.refit_actor._CONTROL_CALL_TIMEOUT_S#
120.0
- nemo_rl.weight_sync.refit_actor._noop_shutdown() bool#
- nemo_rl.weight_sync.refit_actor._disarm_owner(owner: Any) None#
Disarm a generation or policy copy and the cluster its worker group holds.
- nemo_rl.weight_sync.refit_actor._disarm_generation(generation: Any) None#
Disarm the RefitActor’s copy of the generation object.
- nemo_rl.weight_sync.refit_actor._disarm_synchronizer(
- synchronizer: nemo_rl.weight_sync.refit_actor.NcclWeightSynchronizer,
Disarm the copies an NCCL synchronizer brings into the RefitActor.
- class nemo_rl.weight_sync.refit_actor.RefitActorStatus#
What the controller reads back from the RefitActor between refits.
- is_stale: bool#
None
- built_membership: Optional[nemo_rl.weight_sync.membership.RefitMembership]#
None
- class nemo_rl.weight_sync.refit_actor.RefitActor(
- synchronizer: nemo_rl.weight_sync.refit_actor.NcclWeightSynchronizer,
Owns the weight synchronizer;
statusanswers while a refit is stuck.Initialization
- ping() bool#
- update_generation(generation: Any) None#
- init_communicator() Optional[nemo_rl.weight_sync.membership.RefitMembership]#
- sync_weights(
- kv_scales: Optional[dict[str, float]],
- reconcile_communicator(
- absent_shards: collections.abc.Sequence[int],
- force: bool,
- shutdown() None#
- class nemo_rl.weight_sync.refit_actor.RemoteWeightSynchronizer(
- synchronizer: nemo_rl.weight_sync.interfaces.WeightSynchronizer,
- generation: Any,
Bases:
nemo_rl.weight_sync.interfaces.WeightSynchronizerController-side
WeightSynchronizerthat forwards each call to aRefitActor.- Parameters:
synchronizer – an initialized NCCL synchronizer (the config check guarantees one), shipped to the actor.
generation – the controller’s generation object that
synchronizerwas built over.
Initialization
- attach(generation: Any) None#
Bind a proxy unpickled in another process to that process’s generation.
- raise_if_actor_died(error: BaseException) None#
Raise RuntimeError if the RefitActor is gone; a worker’s death passes through.
- _get(
- ref: ray.ObjectRef,
- timeout: Optional[float] = _CONTROL_CALL_TIMEOUT_S,
- _require_generation() Any#
- _worker_identity() tuple[int, ...]#
- _send_generation_if_changed() None#
- _mirror_membership(
- built: Optional[nemo_rl.weight_sync.membership.RefitMembership],
- submit_sync_weights(
- kv_scales: Optional[dict[str, float]] = None,
Start a refit in the actor and return its ObjectRef, for an async caller.
- sync_weights(
- *,
- timer: Optional[nemo_rl.utils.timer.Timer] = None,
- kv_scales: Optional[dict[str, float]] = None,
- property is_stale: bool#
- init_communicator() None#
- reconcile_communicator(
- absent_shards: collections.abc.Sequence[int],
- force: bool = False,
- shutdown() None#
- __getstate__() dict[str, Any]#
- __setstate__(state: dict[str, Any]) None#