nemo_gym.comparison.runner

End-to-end execution of gym eval compare.

Module Contents

Functions

Name	Description
`build_comparison_result`	Load both sides, pick the agent(s), and assemble the full comparison.
`invoked_command`	The `gym eval compare` invocation, for provenance in the report.
`resolve_output_dir`	`--output-dir`, defaulting to the candidate run’s own directory.
`run_comparison`	Build the comparison and write its report artifacts.

API

nemo_gym.comparison.runner.build_comparison_result(
    config: nemo_gym.comparison.schema.ComparisonConfig,
    command: str
) -> nemo_gym.comparison.schema.ComparisonResult

Load both sides, pick the agent(s), and assemble the full comparison.

nemo_gym.comparison.runner.invoked_command() -> str

The gym eval compare invocation, for provenance in the report.

nemo_gym.comparison.runner.resolve_output_dir(
    config: nemo_gym.comparison.schema.ComparisonConfig
) -> pathlib.Path

--output-dir, defaulting to the candidate run’s own directory.

The default resolves the candidate path the same way loading does, so the report lands next to the metrics file that was actually read rather than at a same-named path under the cwd.

nemo_gym.comparison.runner.run_comparison(
    config: nemo_gym.comparison.schema.ComparisonConfig,
    command: str
) -> typing.Tuple[nemo_gym.comparison.schema.ComparisonResult, typing.List[pathlib.Path]]

Build the comparison and write its report artifacts.