nemo_microservices.types.evaluation.config_create_params

Module Contents

Classes

ConfigCreateParams

API

class nemo_microservices.types.evaluation.config_create_params.ConfigCreateParams

Bases: typing_extensions.TypedDict

custom_fields: Dict[str, object]

None

A set of custom fields that the user can define and use for various purposes.

description: str

None

The description of the entity.

groups: Dict[str, nemo_microservices.types.group_config_param.GroupConfigParam]

None

Evaluation tasks belonging to the evaluation.

name: str

None

The name of the entity.

Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.

namespace: str

None

The namespace of the entity.

This can be missing for namespace entities or in deployments that don’t use namespaces.

ownership: nemo_microservices.types.shared_params.ownership.Ownership

None

Information about ownership of an entity.

If the entity is a namespace, the access_policies will typically apply to all entities inside the namespace.

params: nemo_microservices.types.evaluation_params_param.EvaluationParamsParam

None

Global parameters for an evaluation.

project: str

None

The URN of the project associated with this entity.

tasks: Dict[str, nemo_microservices.types.task_config_param.TaskConfigParam]

None

Evaluation tasks belonging to the evaluation.

type: typing_extensions.Required[str]

None

The type of the evaluation, e.g., ‘mmlu’, ‘big_code’.For custom evaluations, this is set to custom.