nemo_rl.experience.rollout_manager#
Module Contents#
Classes#
Recovery-ledger hooks around one physical Gym |
|
Fresh identities for rows a broken Gym stream dropped. |
|
How :meth: |
|
Retry budgets for one prompt, resolved from |
|
Counters describing what the retry policy has been doing. |
|
Deadlines for the blocking waits inside one rollout. |
|
Live request deadlines shared by independently pausable subsystems. |
|
|
|
Manages per-prompt multi-turn rollouts, producing a PromptGroupRecord per call. |
|
Manages per-prompt NeMo-Gym rollouts, producing a PromptGroupRecord per call. |
|
Bind one prompt group’s ledger attempts to its physical Gym submissions. |
|
Routes to AsyncRolloutImpl (native async) or AsyncNemoGymRolloutImpl (NeMo-Gym), and pushes results to a TQReplayBuffer. |
Functions#
Whether an error, including a rollback ExceptionGroup, is post-write. |
|
Return the best available namespace for NeMo-Gym rollout metrics. |
|
Wrap a generation error in the typed failure its class implies. |
|
Gather coroutines, cancelling the remainder as soon as one fails. |
Data#
API#
- nemo_rl.experience.rollout_manager.TokenizerType#
None
- nemo_rl.experience.rollout_manager.RolloutCompletionCallback#
None
- nemo_rl.experience.rollout_manager._contains_post_write_enrichment_error(error: BaseException) bool#
Whether an error, including a rollback ExceptionGroup, is post-write.
- class nemo_rl.experience.rollout_manager.RolloutDispatchRecorder#
Bases:
typing.ProtocolRecovery-ledger hooks around one physical Gym
/runsubmission.A ledger attempt is
DISPATCHEDonly while a/runis actually on its way to Gym. Marking it before the dispatch gate opened left an attempt that a checkpoint counted as Gym-owned, yet that could not reach Gym until the checkpoint released the gate, so the checkpoint drain could only time out.- submitting(
- generation_indices: collections.abc.Sequence[int],
Hold the ledger for one submission; call the yielded hook right after it.
Entered inside the dispatch admission, so a checkpoint cannot close the gate between the submission and the mark. The yielded hook is synchronous so nothing can interleave once the RPC is in flight.
- async refused(generation_index: int) None#
Unwind one attempt Gym refused because a checkpoint closed admission.
- async replace_dropped(
- generation_indices: collections.abc.Sequence[int],
Give rows a broken stream left running in Gym, or Gym failed, fresh attempts.
Returns None once the group was discarded. The abandoned attempts wait for the next checkpoint to retire them.
- class nemo_rl.experience.rollout_manager.ReplacedRows#
Fresh identities for rows a broken Gym stream dropped.
- rollout_ids: collections.abc.Mapping[int, str]#
None
- group_attempt: int#
None
- nemo_rl.experience.rollout_manager._PARKED_REDISPATCH_BACKOFF_S#
0.5
- nemo_rl.experience.rollout_manager._MAX_PARKED_REDISPATCH_BACKOFF_S#
5.0
- nemo_rl.experience.rollout_manager._nemo_gym_metric_namespace(
- row: collections.abc.Mapping[str, Any],
Return the best available namespace for NeMo-Gym rollout metrics.
- class nemo_rl.experience.rollout_manager.RolloutOutcome#
Bases:
str,enum.EnumHow :meth:
RolloutManager.generate_and_pushfinished for one prompt.Initialization
Initialize self. See help(type(self)) for accurate signature.
- COMMITTED#
‘committed’
- SKIPPED#
‘skipped’
- FILTERED#
‘filtered’
- class nemo_rl.experience.rollout_manager.RolloutRetryPolicy#
Retry budgets for one prompt, resolved from
async_rl.rollout_failure.The attempt budgets are required. They previously defaulted to 1/1/1, which contradicted
RolloutFailureConfig’s 5/2/3 and put a second set of defaults in the codebase – a reader here came away believing the shipped budget was 1. The only place a retry default lives isRolloutFailureConfig; callers that need the historical no-retry behaviour ask for it by name via :meth:single_attempt.- max_infra_attempts: int#
None
- max_data_attempts: int#
None
- max_gym_row_attempts: int#
None
- backoff_base_s: float#
1.0
- max_backoff_s: float#
30.0
- max_skipped_prompts: int#
0
- max_consecutive_dropped_prompts: int#
0
- classmethod single_attempt(
- **overrides: Any,
The historical no-retry policy, with optional overrides.
An explicit choice for callers constructing a
RolloutManagerdirectly, who must not silently gain retries – not a second set of defaults.
- __post_init__() None#
- backoff_for(attempt: int) float#
Return the delay before infra attempt
attempt+ 1 (1-based attempts).
- class nemo_rl.experience.rollout_manager.RolloutStats#
Counters describing what the retry policy has been doing.
Read by the SingleController for logging. A stall or a rising redispatch count is the only externally visible sign that the fleet is degrading, so these are not optional bookkeeping.
- committed: int#
0
- skipped: int#
0
- zero_variance_groups: int#
0
- redispatches_by_reason: dict[str, int]#
‘field(…)’
- data_retries_by_reason: dict[str, int]#
‘field(…)’
- data_failures_by_reason: dict[str, int]#
‘field(…)’
- infra_drops_by_reason: dict[str, int]#
‘field(…)’
- max_consecutive_infra_drops: int#
0
- gym_row_redispatches: int#
0
- gym_admission_closed_rows: int#
0
- record_redispatch(reason: str) None#
- record_data_retry(reason: str) None#
- record_data_failure(reason: str) None#
- record_infra_drop(reason: str, consecutive: int) None#
- record_gym_row_redispatch(rows: int = 1) None#
- record_gym_admission_closed(rows: int = 1) None#
- as_metrics() dict[str, float]#
Flatten into a metric dict for the SingleController logger.
- class nemo_rl.experience.rollout_manager.RolloutTimeouts#
Deadlines for the blocking waits inside one rollout.
Resolved from
async_rl.rollout_failure.nemo_gym.rollout_timeout_sandasync_rl.rollout_failure.native.{generation,env}_timeout_s, which own the user-facing defaults.Nonemeans no deadline, reproducing the historical behaviour of waiting indefinitely.- rollout_s: Optional[float]#
None
- generation_s: Optional[float]#
None
- env_s: Optional[float]#
None
- nemo_rl.experience.rollout_manager._classify_generation_failure(
- exc: Exception,
- *,
- prompt_idx: Any,
- traj_idx: int,
Wrap a generation error in the typed failure its class implies.
The original exception is preserved as
__cause__; the prompt and trajectory coordinates are attached because a raw generation traceback does not say which rollout it belonged to.Any Ray-boundary error is infrastructure here, by context. An exception raised inside a still-living generation worker – vLLM
EngineDeadError, a CUDA OOM – arrives as a bareRayTaskErrorwhose cause the boundary degraded, soclassify_rollout_failurecan only fall through to DATA and the prompt gets two attempts instead of the five the fleet-failure path is built for.Scoped to this call site rather than widened in
classify_rollout_failure, deliberately. Globally, “unrecognized means DATA” is the right default: it is about exceptions we can inspect and do not recognize, and flipping it would retry genuine bugs everywhere. Here we have information the classifier does not – this exception came from a generation RPC, so “that shard could not serve it” is the correct reading whatever the destroyed cause was, and re-dispatching to another shard is exactly the right response. A real bug in the worker still surfaces, chained, once the bounded infra budget runs out.This also makes the two paths agree: part 2/4’s
_generate_on_shardalready mapsray.exceptions.RayErrortoGenerationUnavailable, so without this the same exception classified INFRA there and DATA here.- Parameters:
exc – The exception raised while generating a turn.
prompt_idx – Index of the prompt whose rollout failed.
traj_idx – Index of the failing generation within the prompt group.
- Returns:
GenerationUnavailablefor infrastructure failures (retriable on another shard),RolloutDataFailureotherwise.
- async nemo_rl.experience.rollout_manager._gather_cancelling_siblings(
- coros: list[Any],
Gather coroutines, cancelling the remainder as soon as one fails.
asyncio.gatherpropagates the first exception but leaves the other awaitables running detached. On the rollout path those keep occupying generation capacity for a prompt group whose result is already being discarded, so they are cancelled and drained before unwinding.- Parameters:
coros – Coroutines to run concurrently.
- Returns:
Their results, in input order.
- class nemo_rl.experience.rollout_manager.RequestDeadlineRegistry#
Live request deadlines shared by independently pausable subsystems.
Initialization
- _DEFAULT_HOLDER#
‘generation_stand_down’
- property suspended: bool#
Whether at least one subsystem still owns the deadline pause.
- add(deadline: nemo_rl.experience.rollout_manager._Deadline) None#
- discard(
- deadline: nemo_rl.experience.rollout_manager._Deadline,
- suspend(holder: str = _DEFAULT_HOLDER) None#
Pause all deadline clocks for one idempotent owner.
- resume(holder: str = _DEFAULT_HOLDER) None#
Release one owner and resume clocks after the final owner leaves.
- class nemo_rl.experience.rollout_manager._Deadline(
- seconds: Optional[float],
- description: str,
- registry: Optional[nemo_rl.experience.rollout_manager.RequestDeadlineRegistry] = None,
asyncio.timeoutthat reports expiry as a typed :class:RolloutTimeout.A bare
asyncio.timeoutsurfaces expiry asTimeoutError, which is indistinguishable from aTimeoutErrorraised by the wrapped code itself. This consultsexpired()so only a real deadline breach is relabelled, and anything else propagates untouched.seconds=Nonedisables the deadline, matchingasyncio.timeoutsemantics.registryputs the deadline clock in units of inference clock-time, not wall clock-time: When a colocated engine is suspended for training, inference deadlines should not tick down.Initialization
- async __aenter__() nemo_rl.experience.rollout_manager._Deadline#
- async __aexit__(exc_type, exc, tb) Optional[bool]#
- suspend() None#
Disarm the clock, banking whatever budget is left.
- resume() None#
Re-arm the clock with the banked budget.
- class nemo_rl.experience.rollout_manager.AsyncRolloutImpl(
- tokenizer: nemo_rl.experience.rollout_manager.TokenizerType,
- task_to_env: dict[str, nemo_rl.environments.interfaces.EnvironmentInterface],
- num_generations_per_prompt: int,
- max_seq_len: int,
- max_rollout_turns: int,
- policy_generation: nemo_rl.models.generation.interfaces.GenerationInterface,
- timeouts: nemo_rl.experience.rollout_manager.RolloutTimeouts = RolloutTimeouts(),
- deadline_registry: Optional[nemo_rl.experience.rollout_manager.RequestDeadlineRegistry] = None,
- **kwargs: Any,
Manages per-prompt multi-turn rollouts, producing a PromptGroupRecord per call.
Each run_rollout takes one prompt and returns num_generations_per_prompt completions generated concurrently via asyncio.gather.
Initialization
- async run_rollout(
- input_sample: nemo_rl.data.interfaces.DatumSpec,
- *,
- rollout_ids: Optional[list[str]] = None,
- generation_indices: Optional[list[int]] = None,
- on_completion: Optional[nemo_rl.experience.rollout_manager.RolloutCompletionCallback] = None,
- recovery_granularity: nemo_rl.experience.rollout_recovery.RecoveryGranularity = RecoveryGranularity.SIBLING,
- environment_instance_id: Optional[str] = None,
- dispatch_recorder: Optional[nemo_rl.experience.rollout_manager.RolloutDispatchRecorder] = None,
Run num_generations_per_prompt rollouts for one prompt.
- Parameters:
input_sample – A single prompt (one DatumSpec entry).
rollout_ids – Unsupported here — token capture is NeMo-Gym only.
- Returns:
PromptGroupRecord with num_generations_per_prompt completions.
- async _run_single_rollout(
- input_sample: nemo_rl.data.interfaces.DatumSpec,
- traj_idx: int,
Run one multi-turn rollout for a single generation index.
- async _generate_response(
- message_log: list[dict],
- stop_strings: list[str] | None,
- *,
- native_generation_data: dict[str, Any] | None = None,
Generate a single-turn response for one sample.
- Returns:
Tuple of (assistant_message, input_lengths, gen_metrics)
- _aggregate_rollout_metrics(
- completions: list[nemo_rl.experience.interfaces.Completion],
- all_sample_metrics: list[dict],
Aggregate per-sample metrics across all completions.
- class nemo_rl.experience.rollout_manager.AsyncNemoGymRolloutImpl(
- tokenizer: nemo_rl.experience.rollout_manager.TokenizerType,
- task_to_env: dict[str, nemo_rl.environments.interfaces.EnvironmentInterface],
- num_generations_per_prompt: int,
- max_seq_len: int,
- max_rollout_turns: int,
- generation_config: nemo_rl.models.generation.interfaces.GenerationConfig,
- mask_env_flagged_samples: bool = True,
- reward_penalty_config: Optional[dict[str, Any]] = None,
- timeouts: Optional[nemo_rl.experience.rollout_manager.RolloutTimeouts] = None,
- deadline_registry: Optional[nemo_rl.experience.rollout_manager.RequestDeadlineRegistry] = None,
- retry_policy: Optional[nemo_rl.experience.rollout_manager.RolloutRetryPolicy] = None,
- stats: Optional[nemo_rl.experience.rollout_manager.RolloutStats] = None,
- effort_config: Optional[nemo_rl.experience.rollouts.EffortLevelsConfig] = None,
- log_full_result_tables: bool = False,
- forest_training: bool = False,
- length_penalty_config: nemo_rl.utils.length_penalty.LengthPenaltyConfig | None = None,
- **kwargs: Any,
Manages per-prompt NeMo-Gym rollouts, producing a PromptGroupRecord per call.
Each run_rollout takes one prompt and returns num_generations_per_prompt completions batched through a single NeMo-Gym run_rollouts call.
Initialization
- _forest_training: bool#
False
- get_next_nemo_gym_task_index() int#
Return the next attempt ID for checkpointing.
- set_next_nemo_gym_task_index(value: int) None#
Restore the next attempt ID before dispatching rollouts.
- _take_nemo_gym_task_index() int#
- set_rollout_dispatch_admission_gate( ) None#
Bind the controller-owned dispatch barrier of Gym checkpoint coordination.
A
gym_dispatchsection is held only while the Ray streaming RPC is submitted. The returned stream is consumed after it is released, so a checkpoint of this barrier does not wait for generation, completion callbacks, finalization, or TQ writes.
- async run_rollout(
- input_sample: nemo_rl.data.interfaces.DatumSpec,
- *,
- rollout_ids: Optional[list[str]] = None,
- generation_indices: Optional[list[int]] = None,
- on_completion: Optional[nemo_rl.experience.rollout_manager.RolloutCompletionCallback] = None,
- recovery_granularity: nemo_rl.experience.rollout_recovery.RecoveryGranularity = RecoveryGranularity.SIBLING,
- environment_instance_id: Optional[str] = None,
- dispatch_recorder: Optional[nemo_rl.experience.rollout_manager.RolloutDispatchRecorder] = None,
Run num_generations_per_prompt rollouts for one prompt.
- Parameters:
input_sample – A single prompt (one DatumSpec entry).
rollout_ids – Token-capture mode: gate-registered rollout ids, one per generation, riding each row’s run body as the opaque
_ng_rollout_idkey (agents stamp /ng-rollout/from it; zero agent changes). dispatch_recorder – Recovery-ledger hooks that mark each submission dispatched and unwind rows Gym refuses at checkpoint admission.
- Returns:
PromptGroupRecord with num_generations_per_prompt completions.
- _validate_init_params() None#
Validate initialization parameters.
- _build_inputs(
- input_sample: nemo_rl.data.interfaces.DatumSpec,
- *,
- task_index: int,
- rollout_ids: Optional[list[str]] = None,
- generation_indices: Optional[list[int]] = None,
Build N row dicts from input_sample, applying generation config params.
- async _stream_rows(
- nemo_gym_env: Any,
- pending: list[dict],
- results: list[Optional[dict]],
- shaping_by_rowidx: list[Optional[nemo_rl.experience.rollouts._EffortShapingMetrics]],
- total_rows: int,
- timer_prefix: str,
- on_completion: Optional[nemo_rl.experience.rollout_manager.RolloutCompletionCallback] = None,
- dispatch_recorder: Optional[nemo_rl.experience.rollout_manager.RolloutDispatchRecorder] = None,
- admission_closed_rows: Optional[set[int]] = None,
- recovery_granularity: nemo_rl.experience.rollout_recovery.RecoveryGranularity = RecoveryGranularity.SIBLING,
Dispatch
pendingrows and fill their slots inresultsas they land.A row Gym refuses at checkpoint admission is re-sent on its own once dispatch admission reopens, while the other rows keep streaming.
- Parameters:
nemo_gym_env – The NeMo-Gym environment actor handle.
pending – Rows still awaiting a result; each carries its original
_rowidx.results – Full-length result list, mutated in place.
shaping_by_rowidx – Per-row shaping metrics, populated before a completion can be published to the recovery ledger.
total_rows – Size of the original prompt group, used to validate row indices.
timer_prefix – Timer namespace forwarded to the environment.
dispatch_recorder – Marks the submission dispatched in the recovery ledger and unwinds rows Gym refuses at checkpoint admission.
admission_closed_rows – Rows Gym refused because a checkpoint had admission closed and that wait for their re-send; required with
dispatch_recorder.recovery_granularity – Whether a failed row is re-sent on its own or its whole prompt group is replaced.
- Returns:
The environment’s timing metrics, or None if the stream ended without them.
- Raises:
A row's transport-shaped failure only once the stream has ended, and a –
data failure, or any row failure under prompt-group recovery, at once. –
- async _replace_unreturned_rows(
- pending: list[dict],
- results: list[Optional[dict]],
- admission_closed_rows: set[int],
- *,
- dispatch_recorder: Optional[nemo_rl.experience.rollout_manager.RolloutDispatchRecorder],
Re-stamp the rows a stream ended without with fresh rollout IDs.
Gym runs every row of a
/runas its own task, and an ended stream cancels none of them. A re-send under the same identity would run two copies of one episode. The old episodes keep running until the next checkpoint retires them.
- async _run_rollouts(
- inputs: list[dict],
- timer: nemo_rl.utils.timer.Timer,
- timer_prefix: str,
- *,
- on_completion: Optional[nemo_rl.experience.rollout_manager.RolloutCompletionCallback] = None,
- recovery_granularity: nemo_rl.experience.rollout_recovery.RecoveryGranularity = RecoveryGranularity.SIBLING,
- gym_instance_id: Optional[str] = None,
- dispatch_recorder: Optional[nemo_rl.experience.rollout_manager.RolloutDispatchRecorder] = None,
Dispatch rows to NeMo-Gym; return completions, prompt, and metrics.
Sibling recovery re-dispatches only rows that never arrive. Prompt-group recovery performs one physical Gym dispatch here and delegates a complete cohort replacement to the outer recovery loop.
- select_checkpoint_instance(
- input_sample: nemo_rl.data.interfaces.DatumSpec,
Choose and return the durable Gym actor identity for one prompt group.
- _results_to_completions(
- results: list[dict],
Apply configured penalties and convert a Gym result batch.
Receipt-mode (token-capture) results are token-free — the message_log is empty and the canonical row is rebuilt by the finalizer from staged deltas — so they skip tensorization, truncation, and the token/text reward penalties; the receipt and rollout id ride env_extras for the finalize step.
- _compute_reward_penalty_metrics(
- penalty_counts: dict[str, int],
- num_results: int,
Return enabled penalty rates using the legacy Gym metric names.
- _compute_rollout_metrics(
- completions: list[nemo_rl.experience.interfaces.Completion],
- agent_name: str,
- full_results: Optional[list[dict[str, Any]]] = None,
Aggregate per-sample and per-agent metrics.
full_resultsare the Gym results the completions were built from, aligned with them. When given, the per-agent metrics aggregate those rather than the completions’ env_extras, so a field the completion conversion drops on purpose (the env mask flag underenv.should_mask_flagged_samples: false) is still reported.
- class nemo_rl.experience.rollout_manager._LedgerDispatchRecorder#
Bind one prompt group’s ledger attempts to its physical Gym submissions.
- manager: RolloutManager#
None
- group_id: str#
None
- environment_instance_id: Optional[str]#
None
- async submitting(
- generation_indices: collections.abc.Sequence[int],
- async refused(generation_index: int) None#
- async replace_dropped(
- generation_indices: collections.abc.Sequence[int],
- class nemo_rl.experience.rollout_manager.RolloutManager(
- tokenizer: nemo_rl.experience.rollout_manager.TokenizerType,
- task_to_env: dict[str, nemo_rl.environments.interfaces.EnvironmentInterface],
- num_generations_per_prompt: int,
- max_seq_len: int,
- rollout_recovery_config: nemo_rl.algorithms.single_controller_utils.config.RolloutRecoveryConfig,
- max_rollout_turns: int = 1,
- policy_generation: Optional[nemo_rl.models.generation.interfaces.GenerationInterface] = None,
- generation_config: Optional[nemo_rl.models.generation.interfaces.GenerationConfig] = None,
- use_nemo_gym: bool = False,
- mask_env_flagged_samples: bool = True,
- reward_penalty_config: Optional[dict[str, Any]] = None,
- tq_buffer: Optional[nemo_rl.algorithms.async_utils.replay_buffer.TQReplayBuffer] = None,
- timeouts: Optional[nemo_rl.experience.rollout_manager.RolloutTimeouts] = None,
- retry_policy: Optional[nemo_rl.experience.rollout_manager.RolloutRetryPolicy] = None,
- effort_config: Optional[nemo_rl.experience.rollouts.EffortLevelsConfig] = None,
- log_full_result_tables: bool = False,
- forest_training: bool = False,
- execution_row_multiple: int = 1,
- filter_zero_variance_groups: bool = False,
- masked_reward_policy: Optional[str] = None,
- overlong_filtering: bool = False,
- length_penalty_config: nemo_rl.utils.length_penalty.LengthPenaltyConfig | None = None,
Routes to AsyncRolloutImpl (native async) or AsyncNemoGymRolloutImpl (NeMo-Gym), and pushes results to a TQReplayBuffer.
Initialization
- _tally_step_provider: Optional[collections.abc.Callable[[], int]]#
None
- property execution_row_multiple: int#
Return the resolved row divisibility used to publish forest groups.
- property stats: nemo_rl.experience.rollout_manager.RolloutStats#
Counters describing retry/skip activity so far.
- suspend_request_deadlines(
- holder: str = RequestDeadlineRegistry._DEFAULT_HOLDER,
Pause live request-deadline clocks for one idempotent owner.
- resume_request_deadlines(
- holder: str = RequestDeadlineRegistry._DEFAULT_HOLDER,
Resume live clocks after one owner releases its pause.
- property recovery_ledger: nemo_rl.experience.rollout_recovery.RolloutRecoveryLedger#
Return the prompt-group ownership ledger shared with the controller.
- record_finalizer_dropped_prompt() None#
Count a controller-side drop after generation and finalization succeeded.
A group whose valid-row fraction fell below
token_capture.min_valid_fraction_per_groupis not an infra failure of the kind the retry loop above tracks, but it is the same signal for an operator watchingmax_consecutive_dropped_prompts– no rollout got committed for this prompt – so it shares that budget’s counters rather than going uncounted.
- set_data_plane_checkpoint_barrier( ) None#
Join streamed sibling transitions to the SC snapshot barrier.
- set_rollout_dispatch_admission_gate( ) None#
Bind the narrow Gym RPC barrier used while a checkpoint is prepared.
- async _recovery_mutation(
- kind: nemo_rl.algorithms.async_utils.replay_buffer.CheckpointMutationKind = 'recovery_retries',
Serialize short lineage transitions with native TQ snapshots.
- telemetry_snapshot() dict[str, int]#
Return cumulative committed-publication and recovery counters.
- record_canonical_publication(output_tokens: int) None#
Count one prompt group after its canonical TQ commit succeeds.
- record_recovery_siblings(*, reused: int, redispatched: int) None#
Count sibling work avoided and repeated after a process restart.
- reserve_prompt_group(
- cut: nemo_rl.algorithms.async_utils.replay_buffer.DataPlaneMutationCut,
- input_sample: nemo_rl.data.interfaces.DatumSpec,
- *,
- target_step: Optional[int],
- admitted: bool = True,
- admission_id: Optional[str] = None,
Own a prompt before controller dispatch can yield or checkpoint.
- mark_prompt_group_admitted(
- cut: nemo_rl.algorithms.async_utils.replay_buffer.DataPlaneMutationCut,
- group_id: str,
- *,
- target_step: Optional[int],
Attach sampler admission state to a pre-admission reservation.
- discard_prompt_group(
- cut: nemo_rl.algorithms.async_utils.replay_buffer.DataPlaneMutationCut,
- group_id: str,
Release a reservation that will intentionally never be dispatched.
- set_weight_version(version: int) None#
Set the weight_version used for rollout tags.
- Parameters:
version – Trainer weight version to stamp on future rollout tags.
- get_next_nemo_gym_task_index() int#
Return the Gym attempt counter, or zero for native rollouts.
- set_next_nemo_gym_task_index(value: int) None#
Restore the attempt counter for Gym rollouts only.
- set_tally_step_provider(
- provider: collections.abc.Callable[[], int],
Set how a group with no target step finds the step it is tallied under.
- Parameters:
provider – Zero-arg accessor for the step currently being assembled.
- async run_rollout(
- input_sample: nemo_rl.data.interfaces.DatumSpec,
- *,
- rollout_ids: Optional[list[str]] = None,
- generation_indices: Optional[list[int]] = None,
- on_completion: Optional[nemo_rl.experience.rollout_manager.RolloutCompletionCallback] = None,
- recovery_granularity: nemo_rl.experience.rollout_recovery.RecoveryGranularity = RecoveryGranularity.SIBLING,
- environment_instance_id: Optional[str] = None,
- dispatch_recorder: Optional[nemo_rl.experience.rollout_manager.RolloutDispatchRecorder] = None,
- _voting_rewards( ) list[float]#
This group’s voting rewards under the run’s masking policy.
- _is_zero_variance_finalized( ) bool#
Whether a finalized capture group’s voters leave every row zero advantage.
- record_group_rewards(
- target_step: Optional[int],
- rewards: collections.abc.Sequence[float],
- *,
- replacement_index: int,
- zero_variance: bool,
Record group rewards under its stamp or the supplied tally step.
- drain_step_reward_metrics(step: int) dict[str, float]#
Report this step’s rewards and retire this and older step tallies.
- async generate_and_push(
- input_sample: nemo_rl.data.interfaces.DatumSpec,
- *,
- target_step: Optional[int] = None,
- inflight_registry: Optional[dict[str, tuple[asyncio.Task[None], int]]] = None,
- lineage_group_id: Optional[str] = None,
- replacement_index: int = 0,
Roll out one prompt and commit it, re-dispatching on infrastructure failure.
No prompt is discarded for infrastructure reasons. An infra failure means the fleet is unwell, not the prompt, so the attempt is retried – and because each retry re-enters generation-shard selection, it naturally lands somewhere else without this method needing to know anything about shard health. Exhausting the infra budget therefore means the failure follows the prompt across the whole fleet, which is reported as fleet-wide failure rather than absorbed.
Deterministic failures get their own, much smaller budget: another shard would reject the prompt identically, so retrying mostly burns time. One retry is still worth taking because a shard under memory pressure can return an empty generation that looks deterministic and is not.
- Parameters:
input_sample – A single prompt (one DatumSpec entry).
target_step – Training step this rollout targets, stamped on the buffer slot;
Nonewhen the sampler stamps no step.inflight_registry – Optional controller-owned mapping from group ID to its dispatch task and start weight version.
lineage_group_id – Stable group minted by the rollout ledger before dataloader dispatch. TQ records this same ID rather than minting one.
Nonepreserves the ordinary non-checkpointed fresh-ID retry path.replacement_index – 0 for the originally scheduled prompt, N for its N-th replacement after a group was dropped. Only used to separate first attempts in the per-step reward metrics.
- Returns:
COMMITTEDwhen the group reached the buffer, orSKIPPEDwhen the prompt was given up on within a budget: its data budget withinmax_skipped_prompts, or its infra budget withinmax_consecutive_dropped_prompts. ASKIPPEDprompt committed nothing, so the caller owns both its backpressure permit and the checkpoint-atomic transition from its retained ledger record to either a replacement prompt or the shortfall for the training step it is charged to.FILTEREDwhen dynamic sampling dropped the group before commit because every generation scored the same reward. It carries the same two caller obligations asSKIPPED– the backpressure permit and the ledger transition – but the rollout itself succeeded, so its replacement is paid for out ofgrpo.dynamic_sampling_max_gen_batchesrather than the rollout fault-tolerance budget.- Raises:
RolloutRedispatchExhausted – The infra budget ran out and the fleet has not committed anything since
max_consecutive_dropped_promptsdrops ago.RolloutDataFailure – The data budget ran out beyond
max_skipped_prompts.
- async generate_for_finalization(
- input_sample: nemo_rl.data.interfaces.DatumSpec,
- *,
- target_step: Optional[int] = None,
- inflight_registry: Optional[dict[str, tuple[asyncio.Task[None], int]]] = None,
- lineage_group_id: Optional[str] = None,
Capture siblings with stable lineage and configured retry granularity.
Returns the metadata-only request and the group’s rollout metrics, which are empty unless the successful attempt dispatched every sibling. Returns
Nonewhen infrastructure retries are exhausted within the configured drop budget. The caller then owns the backpressure permit and target-step shortfall.
- async _generate_for_finalization_attempt(
- input_sample: nemo_rl.data.interfaces.DatumSpec,
- *,
- recovery_group_id: str,
- inflight_registry: Optional[dict[str, tuple[asyncio.Task[None], int]]],
Dispatch the current sibling cohort and leave one slot unready.
- async discard_recovery_group(
- cut: nemo_rl.algorithms.async_utils.replay_buffer.DataPlaneMutationCut,
- group_id: str,
Clean known staged rows before intentionally dropping lineage.