nemo_gym.eval.comparison.suite.failures

Read saved run failure accounting without scanning rollout records or attempt history.

Module Contents

Functions

Name	Description
`read_saved_failures`	Read this rollout file’s existing coverage snapshot; absent accounting remains unknown.
`saved_failure_warning`	Summarize positive saved counts or unavailable accounting in one advisory line.

Data

_COUNT_KEYS

API

nemo_gym.eval.comparison.suite.failures.read_saved_failures(
    path: pathlib.Path
) -> nemo_gym.eval.comparison.suite.schema.SuiteFailureSummary | None

Read this rollout file’s existing coverage snapshot; absent accounting remains unknown.

The saved counts cover the entire rollout file, including every agent and benchmark. This reader neither refreshes the snapshot nor interprets historical failure rows.

nemo_gym.eval.comparison.suite.failures.saved_failure_warning(
    summary: nemo_gym.eval.comparison.suite.schema.SuiteFailureSummary
) -> str | None

Summarize positive saved counts or unavailable accounting in one advisory line.

nemo_gym.eval.comparison.suite.failures._COUNT_KEYS = {'total': 'failed', 'final': 'failed_final', 'not_final': 'failed_not_final', 'b...