nemo_gym.rollout_compare.report

Render a CompareResult as the text report and the JSON summary.

Finds the losing side’s verifier output, from a file under the row’s verifier logs directory resolved against the roots or from a field embedded in the row, tails it, and lays out the header, the category table and the flip, masked and missing sections. summary_dict is the JSON shape the command writes.

Module Contents

Functions

Name	Description
`_flip_block`	-
`_losing_row`	The worst row on the losing side (meaningful when a side has repeats).
`_resolve_logs_path`	-
`_roots_for`	-
`render_report`	-
`summary_dict`	-
`tail_lines`	-
`verifier_output`	(text, source) for a row’s verifier stdout: the file under `verifier_logs_dir` when it exists, else the

API

nemo_gym.rollout_compare.report._flip_block(
    task: nemo_gym.rollout_compare.schema.TaskComparison,
    roots: nemo_gym.rollout_compare.schema.Roots,
    tail: int
) -> list[str]

nemo_gym.rollout_compare.report._losing_row(
    side: nemo_gym.rollout_compare.schema.TaskSide
) -> dict[str, typing.Any]

The worst row on the losing side (meaningful when a side has repeats).

nemo_gym.rollout_compare.report._resolve_logs_path(
    logs_dir: str,
    roots: list[pathlib.Path]
) -> typing.Optional[pathlib.Path]

nemo_gym.rollout_compare.report._roots_for(
    roots: nemo_gym.rollout_compare.schema.Roots,
    side: nemo_gym.rollout_compare.schema.Side
) -> list[pathlib.Path]

nemo_gym.rollout_compare.render_report(
    result: nemo_gym.rollout_compare.schema.CompareResult,
    roots: nemo_gym.rollout_compare.schema.Roots,
    tail: int = 15
) -> str

nemo_gym.rollout_compare.summary_dict(
    result: nemo_gym.rollout_compare.schema.CompareResult
) -> dict[str, typing.Any]

nemo_gym.rollout_compare.tail_lines(
    text: str,
    count: int
) -> list[str]

nemo_gym.rollout_compare.verifier_output(
    row: dict[str, typing.Any],
    roots: list[pathlib.Path]
) -> tuple[typing.Optional[str], str]

(text, source) for a row’s verifier stdout: the file under verifier_logs_dir when it exists, else the text embedded in the row; (None, reason) when neither is available.