nemo_gym.comparison.report

Rendering a ComparisonResult: the markdown report, its JSON twin, and the terminal recap.

The renderers are pure functions of the result object, so what the statistics layer eventually adds to the schema is the only thing that changes what they print.

Module Contents

Functions

Name	Description
`_candidate_labels`	-
`_flip_section`	-
`_fmt`	-
`_fmt_ci`	-
`_fmt_delta_cell`	-
`_fmt_rewards`	-
`_fmt_signed`	-
`_metric_table`	-
`_run_summary_table`	-
`_select_shown_flips`	At most `MAX_FLIPS_SHOWN` per direction, keeping the ordering `build_flip_summary` chose.
`_table`	-
`render_key_metrics_tables`	One Rich table of key metrics per compared agent, for the terminal recap.
`render_markdown`	-
`summary_lines`	The short stdout recap printed after the files are written.
`write_reports`	Write the requested artifacts into `output_dir`, returning the paths written.

Data

CI_FOOTNOTE

DELTA_HEADER

JSON_REPORT_NAME

MARKDOWN_REPORT_NAME

MISSING

API

nemo_gym.comparison.report._candidate_labels(
    result: nemo_gym.comparison.schema.ComparisonResult
) -> typing.List[str]

nemo_gym.comparison.report._flip_section(
    summary: nemo_gym.comparison.schema.FlipSummary,
    label: str
) -> typing.List[str]

nemo_gym.comparison.report._fmt(
    value: typing.Optional[float]
) -> str

nemo_gym.comparison.report._fmt_ci(
    value: typing.Optional[nemo_gym.comparison.schema.MetricValue]
) -> str

nemo_gym.comparison.report._fmt_delta_cell(
    candidate
) -> str

nemo_gym.comparison.report._fmt_rewards(
    rewards: typing.Optional[typing.Sequence[float]]
) -> str

nemo_gym.comparison.report._fmt_signed(
    value: typing.Optional[float]
) -> str

nemo_gym.comparison.report._metric_table(
    rows: typing.Sequence[nemo_gym.comparison.schema.MetricRow],
    candidate_labels: typing.Sequence[str]
) -> typing.List[str]

nemo_gym.comparison.report._run_summary_table(
    result: nemo_gym.comparison.schema.ComparisonResult,
    comparison: nemo_gym.comparison.schema.AgentComparison
) -> typing.List[str]

nemo_gym.comparison.report._select_shown_flips(
    summary: nemo_gym.comparison.schema.FlipSummary
) -> typing.List

At most MAX_FLIPS_SHOWN per direction, keeping the ordering build_flip_summary chose.

nemo_gym.comparison.report._table(
    header: typing.Sequence[str],
    alignments: typing.Sequence[str],
    rows: typing.Sequence[typing.Sequence[str]]
) -> typing.List[str]

nemo_gym.comparison.report.render_key_metrics_tables(
    result: nemo_gym.comparison.schema.ComparisonResult
) -> typing.List[rich.table.Table]

One Rich table of key metrics per compared agent, for the terminal recap.

nemo_gym.comparison.report.render_markdown(
    result: nemo_gym.comparison.schema.ComparisonResult
) -> str

nemo_gym.comparison.report.summary_lines(
    result: nemo_gym.comparison.schema.ComparisonResult,
    written: typing.Sequence[pathlib.Path]
) -> typing.Tuple[str, ...]

The short stdout recap printed after the files are written.

nemo_gym.comparison.report.write_reports(
    result: nemo_gym.comparison.schema.ComparisonResult,
    output_dir: pathlib.Path,
    report_format: str
) -> typing.List[pathlib.Path]

Write the requested artifacts into output_dir, returning the paths written.

nemo_gym.comparison.report.CI_FOOTNOTE = f'CI = 95% t-interval of the per-repeat mean across repeats, read verbatim from ...

nemo_gym.comparison.report.DELTA_HEADER = 'Δ (cand − base)'

nemo_gym.comparison.report.JSON_REPORT_NAME = 'compare_report.json'

nemo_gym.comparison.report.MARKDOWN_REPORT_NAME = 'compare_report.md'

nemo_gym.comparison.report.MISSING = '—'