nemo_gym.rollout_compare.rows

Read rollout rows and what each row says about itself.

Strict per-line JSONL parsing of the main file and the failures sidecar, with the last failure per attempt folded in as a masked row, and the per-row signals: the mask reason, the measured failure kind, the agent error and the strict numeric reward. summarize_side reduces one side’s rows for a task to a TaskSide.

Module Contents

Functions

Name	Description
`_attempt_key`	The collector’s (task index, rollout index) identity of a row, or None when the row does not carry it.
`_failure_as_masked_row`	A `<name>_failures.jsonl` record becomes a masked row with no reward.
`_finite_reward_of`	-
`_read_jsonl`	-
`_reward_of`	The row’s reward as a float (None when absent; NaN/inf pass through for `mask_reason` to catch). Booleans
`_row_task_label`	Best-effort task name for an error about one row, before the join key is known.
`agent_error`	An agent-level error the row reports without being masked: the legacy agent’s `error` field or an
`current_failures`	The sidecar records that still stand: the collector writes one record per attempt and a retried rollout
`load_rollouts`	Read a rollouts JSONL plus, when present, its sibling `<name>_failures.jsonl` of masked failures. A failure
`mask_reason`	Why a row does not count: `mask_sample`, no reward, or a NaN/infinite reward.
`measured_failure_kind`	The `failure_kind` of a row that still counts (not masked): a scored failure the report labels but does
`summarize_side`	-

API

nemo_gym.rollout_compare.rows._attempt_key(
    row: dict[str, typing.Any]
) -> typing.Optional[tuple[typing.Any, typing.Any]]

The collector’s (task index, rollout index) identity of a row, or None when the row does not carry it.

nemo_gym.rollout_compare.rows._failure_as_masked_row(
    failure: dict[str, typing.Any]
) -> dict[str, typing.Any]

A <name>_failures.jsonl record becomes a masked row with no reward.

nemo_gym.rollout_compare.rows._finite_reward_of(
    row: dict[str, typing.Any]
) -> typing.Optional[float]

nemo_gym.rollout_compare.rows._read_jsonl(
    path: pathlib.Path
) -> list[dict[str, typing.Any]]

nemo_gym.rollout_compare.rows._reward_of(
    row: dict[str, typing.Any]
) -> typing.Optional[float]

The row’s reward as a float (None when absent; NaN/inf pass through for mask_reason to catch). Booleans and strings are rejected rather than coerced: a "1.0" string means the producer is broken.

nemo_gym.rollout_compare.rows._row_task_label(
    row: dict[str, typing.Any]
) -> str

Best-effort task name for an error about one row, before the join key is known.

nemo_gym.rollout_compare.agent_error(
    row: dict[str, typing.Any]
) -> typing.Optional[str]

An agent-level error the row reports without being masked: the legacy agent’s error field or an *error_type entry in the response metadata (e.g. Terminus’s terminus2_error_type).

nemo_gym.rollout_compare.current_failures(
    failures: list[dict[str, typing.Any]],
    scored: list[dict[str, typing.Any]]
) -> list[dict[str, typing.Any]]

The sidecar records that still stand: the collector writes one record per attempt and a retried rollout can end as a scored row in the main file, so only the last record per (task index, rollout index) counts and none whose rollout was scored after all. Records without the indices are kept as they are.

nemo_gym.rollout_compare.load_rollouts(
    path: str | pathlib.Path
) -> list[dict[str, typing.Any]]

Read a rollouts JSONL plus, when present, its sibling <name>_failures.jsonl of masked failures. A failure superseded by a later attempt (another sidecar record or a scored row) is left out.

nemo_gym.rollout_compare.mask_reason(
    row: dict[str, typing.Any]
) -> typing.Optional[str]

Why a row does not count: mask_sample, no reward, or a NaN/infinite reward.

A failure_kind alone does not mask: with mask_sample false it labels a measured reward (e.g. the Harbor server’s scored zero for a verifier timeout), which counts like any other reward; see measured_failure_kind.

nemo_gym.rollout_compare.measured_failure_kind(
    row: dict[str, typing.Any]
) -> typing.Optional[str]

The failure_kind of a row that still counts (not masked): a scored failure the report labels but does not set aside, so a flip it causes is a real flip.

nemo_gym.rollout_compare.summarize_side(
    rows: list[dict[str, typing.Any]]
) -> nemo_gym.rollout_compare.schema.TaskSide