nemo_microservices.types.v2.evaluation.evaluation_config_v2_param#
Module Contents#
Classes#
API#
- class nemo_microservices.types.v2.evaluation.evaluation_config_v2_param.EvaluationConfigV2Param#
Bases:
typing_extensions.TypedDict- groups: Dict[str, nemo_microservices.types.group_config_param.GroupConfigParam]#
None
Evaluation groups belonging to the evaluation.
- params: nemo_microservices.types.evaluation_params_param.EvaluationParamsParam#
None
Global parameters for an evaluation.
- tasks: Dict[str, nemo_microservices.types.task_config_param.TaskConfigParam]#
None
Evaluation tasks belonging to the evaluation.
- type: typing_extensions.Required[str]#
None
The type of the evaluation, e.g., ‘mmlu’, ‘big_code’.For custom evaluations, this is set to
custom.