nemo_gym.eval.comparison.planning.schema
nemo_gym.eval.comparison.planning.schema
Strict direct-input manifest and completed report contracts for repeat planning.
Module Contents
Classes
Data
API
Bases: _PlanningModel
Whether baseline repeats are existing measurements or a planned investment.
Bases: BaseNeMoGymCLIConfig
Plan fixed repeat counts from a YAML of normalized repeat standard deviations.
Bases: _PlanningModel
One normalized complete-repeat benchmark score for one model.
For a multi-metric benchmark, normalize and equally average its metrics within each repeat before estimating this score’s SD across repeats. An omitted candidate SD defaults to the baseline SD.
Bases: _PlanningModel
Bases: _PlanningModel
One fixed aggregate planned from direct normalized SD inputs.
benchmark selects one model/benchmark cell, model selects cells of a
single model, and suite selects all declared cells. Every cell has equal
aggregate weight and supplies the SD of one complete-repeat benchmark score.
Bases: _PlanningModel
Completed numerical results and resolved inputs for JSON and Markdown reports.
Bases: _PlanningModel
Bases: BaseModel