nemo_gym.rollout_compare.command

The gym dev compare command surface.

RolloutCompareConfig is the pydantic config whose fields and docstring are the command’s options, and run_compare loads both sides, compares them, resolves the per-side log roots, writes the JSON summary first and then the report or the JSON to stdout, returning the exit code.

Module Contents

Classes

Name	Description
`RolloutCompareConfig`	Compare two rollout JSONL files task by task: count identical, flipped, masked and missing tasks and print the

Functions

Name	Description
`run_compare`	Load both sides, print the report (or the JSON summary with `json_stdout`) and write the JSON file when

API

class nemo_gym.rollout_compare.RolloutCompareConfig()

Bases: BaseNeMoGymCLIConfig

Compare two rollout JSONL files task by task: count identical, flipped, masked and missing tasks and print the losing side’s verifier output for every flip. A report, not a gate: the exit code is 0 for a report and 1 when an input cannot be read or the --json summary cannot be written.

Examples:

# Old server vs Harbor path on the same tasks
gym dev compare old/rollouts.jsonl new/rollouts.jsonl
# Resolve relative `verifier_logs_dir` paths against the Harbor server directory, keep 30 log lines per flip,
# and also write a machine-readable summary
gym dev compare old.jsonl new.jsonl --logs-root environment_servers/harbor --tail 30 --json compare.json
# Join on an explicit field when auto-detection picks the wrong one
gym dev compare old.jsonl new.jsonl --key verifier_metadata.instance_id

json_output

Optional[str]

json_stdout

bool

key

Optional[str]

logs_root

Optional[str]

new_rollouts

str

old_rollouts

str

tail

int

nemo_gym.rollout_compare.run_compare(
    config: nemo_gym.rollout_compare.command.RolloutCompareConfig,
    out = None
) -> int

Load both sides, print the report (or the JSON summary with json_stdout) and write the JSON file when asked. Exit code: 0 for a report, 1 for unreadable input or unwritable output.