nemo_gym.eval.comparison.suite.runner

Orchestrate suite artifact loading, processing, and result provenance.

Module Contents

Functions

Name	Description
`build_suite_comparison`	Build typed results, optionally resolving catalog reward scales through the supplied callback.
`invoked_command`	Record the suite command with its Hydra overrides, masking secret values.

API

nemo_gym.eval.comparison.suite.runner.build_suite_comparison(
    config: nemo_gym.eval.comparison.suite.schema.SuiteComparisonConfig,
    command: str,
    reward_metric_resolver: collections.abc.Callable[[str], nemo_gym.eval.comparison.suite.schema.SuiteMetricResult] | None = None
) -> nemo_gym.eval.comparison.suite.schema.SuiteComparisonResult

Build typed results, optionally resolving catalog reward scales through the supplied callback.

nemo_gym.eval.comparison.suite.runner.invoked_command() -> str

Record the suite command with its Hydra overrides, masking secret values.