nemo_microservices.types.evaluation_config

Module Contents

Classes

EvaluationConfig

API

class nemo_microservices.types.evaluation_config.EvaluationConfig(/, **data: typing.Any)

Bases: nemo_microservices._models.BaseModel

created_at: Optional[datetime.datetime]

None

Timestamp for when the entity was created.

custom_fields: Optional[Dict[str, object]]

None

A set of custom fields that the user can define and use for various purposes.

description: Optional[str]

None

The description of the entity.

groups: Optional[Dict[str, nemo_microservices.types.group_config.GroupConfig]]

None

Evaluation tasks belonging to the evaluation.

id: Optional[str]

None

The ID of the entity.

With the exception of namespaces, this is always a semantically-prefixed base58-encoded uuid4 [-base58(uuid4())].

name: Optional[str]

None

The name of the entity.

Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.

namespace: Optional[str]

None

The namespace of the entity.

This can be missing for namespace entities or in deployments that don’t use namespaces.

ownership: Optional[nemo_microservices.types.shared.ownership.Ownership]

None

Information about ownership of an entity.

If the entity is a namespace, the access_policies will typically apply to all entities inside the namespace.

params: Optional[nemo_microservices.types.evaluation_params.EvaluationParams]

None

Global parameters for an evaluation.

project: Optional[str]

None

The URN of the project associated with this entity.

schema_version: Optional[str]

None

The version of the schema for the object. Internal use only.

tasks: Optional[Dict[str, nemo_microservices.types.task_config.TaskConfig]]

None

Evaluation tasks belonging to the evaluation.

type: str

None

The type of the evaluation, e.g., ‘mmlu’, ‘big_code’.For custom evaluations, this is set to custom.

type_prefix: Optional[str]

None

updated_at: Optional[datetime.datetime]

None

Timestamp for when the entity was last updated.