nemo_microservices.types.v2.evaluation.evaluation_config_v2#
Module Contents#
Classes#
API#
- class nemo_microservices.types.v2.evaluation.evaluation_config_v2.EvaluationConfigV2(/, **data: typing.Any)#
Bases:
nemo_microservices._models.BaseModel- groups: Optional[Dict[str, nemo_microservices.types.group_config.GroupConfig]]#
None
Evaluation groups belonging to the evaluation.
- params: Optional[nemo_microservices.types.evaluation_params.EvaluationParams]#
None
Global parameters for an evaluation.
- tasks: Optional[Dict[str, nemo_microservices.types.task_config.TaskConfig]]#
None
Evaluation tasks belonging to the evaluation.
- type: str#
None
The type of the evaluation, e.g., ‘mmlu’, ‘big_code’.For custom evaluations, this is set to
custom.