nemo_gym.data.tasks.harbor.task_data
nemo_gym.data.tasks.harbor.task_data
The dataset’s task_data.py: the TaskData model every row’s task fields are validated against.
A Gym-authored dataset may ship task_data.py beside dataset.toml, a module exporting one
Pydantic model named TaskData that declares the task-owned fields of its rows. A task’s fields
come from task_data.json in its folder (a parameterized task’s rendered row; a hand-written task
may carry one too) and travel in the row’s task_input.task_data beside the reserved _ng_*
provenance keys. The same check runs at the three points a row crosses a boundary:
- at materialize,
prepare_targetloads the module once per dataset and refuses the whole dataset, naming the offending task, when a task’s fields do not satisfy the model (the folders’task_data.jsonfiles are checked before any image is recorded or built, the loaded tasks again before a row is written); - at seed, the
harborenvironment server validates the row it is handed and refuses the episode (422, not retried) when it does not satisfy the model or differs from the task folder’s; - at collate,
gym dataset collatevalidates every row of the taskset the same way.
Unknown keys fail everywhere: TaskData is the complete description of the fields, whatever its
extra setting, so a typo in a row cannot pass as an optional field, and neither can a key no
field reads (a field’s name where only its alias is accepted, a second spelling of a field already
given): the model is validated with extras forbidden, so pydantic’s own accepted input names decide.
Values are validated strictly, as JSON: a value of another JSON type ("3" for an int, "no"
for a bool) is refused, not coerced, because the row and the verifier read the file’s values as
written, never the model’s. The reserved keys are never part of TaskData and never admitted as
task fields. A dataset without task_data.py is not checked beyond the reserved keys (every hub
dataset).
The module is imported, not parsed: TaskData is a plain Pydantic class and may use whatever the
standard library and Pydantic offer. It is loaded under a module name derived from its path, so two
datasets with the same folder name keep separate modules.
Module Contents
Classes
Functions
Data
API
Bases: TaskDataValidator
The collator’s validator for a Harbor taskset whose dataset ships task_data.py.
A materialized row’s task_input.task_data is two things in one mapping: the framework’s reserved
_ng_* keys, which the environment server’s schema describes, and the task-owned fields beside them,
which belong to the dataset (task_data.json) and to its TaskData when it ships one. The two are
validated separately into one report, as at prepare and seed: the server’s schema sees the reserved keys
alone, and the dataset’s model, if any, the whole task_data (task_fields drops the reserved
keys for it). Fed the whole mapping, the server’s validator would report every task field of an untyped
dataset as unknown and splice a task’s own verifier_metadata mapping apart as the legacy row format
(nemo_gym.data.task_data.normalize_task_fields). A dataset without task_data.py is checked for the
reserved keys only.
Bases: HarborTaskError
task_data.py does not load, or a task’s fields do not satisfy its TaskData.
A loaded task_data.py: where it came from and the model it exports.
TaskData with extra="forbid": the model that validates rows.
Pydantic alone knows which input names a field reads (its name, its alias or validation_alias, the
first of several AliasChoices present, subject to validate_by_name/validate_by_alias). With
extras forbidden it refuses every key no field consumed, so a key that would otherwise be dropped on
the floor while the field takes its default or another spelling’s value is an error, not a silently
unvalidated value: {"difficulty": "x"} against difficulty: int = Field(default=1, alias="level"),
or expected_answer given beside the answer the field read first. The author’s model is left as
it is; this subclass only forbids extras on top of it.
The row’s task fields as an instance of TaskData; raises TaskDataError naming the task.
A top-level key no field of the model reads (strict_model) is refused as undeclared before any other
error is reported. An extra key inside a nested model the author marked extra="forbid" is not a key of
TaskData: it stays a validation error at its location (meta.bogus: Extra inputs are not permitted),
reported beside the row’s other value errors. Whatever else the model raises is that error: a pydantic ValidationError with its
fields spelled out, or anything else an author’s validator lets escape (a KeyError from a lookup, a
TypeError on a value of the wrong type), named with its type. Every caller (prepare, the seed, collate)
handles one exception class, so a validator defect refuses the dataset, fails the episode as terminal or is
reported against the row instead of crashing the command or being retried as a provider failure.
The values are validated strictly, in JSON terms: the fields are what a JSON file holds, so the model reads
them as JSON (model_validate_json with strict=True), and a value of another JSON type is refused,
never coerced. The model returned here is discarded by every caller: the row carries the file’s values as
written, and so does the task_data.json the in-sandbox verifier reads (task.task_data, the same
fields). Lax validation would accept {"flag": "no"} against flag: bool (reading False) and
{"count": "3"} against count: int while the verifier reads the strings, "no" truthy among them.
JSON’s own spellings of a type still pass, as they must for a file to hold them at all: an ISO string for a
datetime, an enum’s value, an integer where a float is declared. A validator that rewrites a value
changes the model, not the row: write the value the verifier should read in the file, and let a validator
refuse rather than normalize.
The key check stays a pass over the fields as Python values, lax, because only there does pydantic’s
extra="forbid" count the one input name a field consumed: fed JSON, it treats every name a field could
read as known, so a field’s name beside its alias, or a second AliasChoices spelling beside the first,
would pass unvalidated. That pass is forced lax (strict=False) because strict_model inherits the
author’s model_config, and an author’s strict=True would otherwise refuse JSON’s spellings there (an
ISO string for a datetime) that the JSON pass, which alone decides the value types, accepts. Both passes
are pydantic’s own; the first also runs the author’s validators.
Refuse a row whose task fields are not the task folder’s: the row was edited after materialization.
The digest the row carries pins the folder; this pins the row to it. Raises TaskDataError.
Validate each folder’s task_data.json against schema before the tasks load; the first violation refuses
the dataset.
task_folders are paths. The fields need no loaded task, so prepare runs this right after the module imports,
before any base image is resolved or any Dockerfile built and pushed for a dataset that is refused as a whole
anyway; check_tasks over the loaded tasks stays the final guard. A file that does not read (not a JSON
object, a reserved key) is left to the loader, which skips that task with the file’s error. Without a schema
nothing is checked.
Validate every loaded task’s fields against schema; the first violation refuses the dataset.
tasks are HarborTask instances, checked in order. Without a
schema nothing is checked: a dataset that ships no task_data.py behaves as before.
Import <dataset_root>/task_data.py; None when the dataset ships none.
Raises TaskDataError when the module does not import, exports no TaskData, or
exports one that is not a Pydantic model class.
The task’s own field values from <task_folder>/task_data.json; {} when the file is absent.
The file must hold a JSON object whose keys are not reserved. Raises TaskDataError otherwise.
The task-owned part of a row’s task_data: everything but the reserved _ng_* keys.