> This page is for version 0.5.1.
> For other versions, use one of these documentation indexes:
> - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt
> - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt
> - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt
> - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt
> - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt
> - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt
> - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt
> - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# nemo_gym.data.tasks.harbor.materialize

Turn loaded tasks into episode rows and the run config that serves them.

## Module Contents

### Classes

| Name                                                                         | Description                                                                                  |
| ---------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| [`HarborTaskInput`](#nemo_gym-data-tasks-harbor-materialize-HarborTaskInput) | A Harbor row's input: one `responses_create_params` for a single-turn task, or one per turn. |
| [`HarborTurnInput`](#nemo_gym-data-tasks-harbor-materialize-HarborTurnInput) | One turn of a multi-step task: the Harbor step's name and the request the agent receives.    |

### Functions

| Name                                                                                                 | Description                                                                                             |
| ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- |
| [`_instance_taskset`](#nemo_gym-data-tasks-harbor-materialize-_instance_taskset)                     | -                                                                                                       |
| [`_turn_params`](#nemo_gym-data-tasks-harbor-materialize-_turn_params)                               | -                                                                                                       |
| [`_validate_task_ids`](#nemo_gym-data-tasks-harbor-materialize-_validate_task_ids)                   | -                                                                                                       |
| [`agent_instance`](#nemo_gym-data-tasks-harbor-materialize-agent_instance)                           | Return an override-compatible agent instance name for the taskset.                                      |
| [`dataset_entry`](#nemo_gym-data-tasks-harbor-materialize-dataset_entry)                             | The `datasets:` entry that declares a taskset's rows, from the dataset's `[gym]` table.                 |
| [`environment_server_instance`](#nemo_gym-data-tasks-harbor-materialize-environment_server_instance) | Return an override-compatible environment instance name for the taskset.                                |
| [`materialize_task`](#nemo_gym-data-tasks-harbor-materialize-materialize_task)                       | One row: the instruction as a user message, the folder digest and the task's fields in `task_data`.     |
| [`rows_metrics_sidecars`](#nemo_gym-data-tasks-harbor-materialize-rows_metrics_sidecars)             | The data processor's per-dataset metrics beside the rows and the conflict file it writes on a mismatch: |
| [`run_config`](#nemo_gym-data-tasks-harbor-materialize-run_config)                                   | The run-config blocks that bind one taskset to the `harbor` environment server.                         |
| [`taskset_mapping`](#nemo_gym-data-tasks-harbor-materialize-taskset_mapping)                         | Where a taskset's folders live and which digest each task had at materialization.                       |
| [`write_rows`](#nemo_gym-data-tasks-harbor-materialize-write_rows)                                   | Write one materialized row per task as JSONL and return the rows.                                       |

### Data

[`AGENT_TIMEOUT_METADATA_KEY`](#nemo_gym-data-tasks-harbor-materialize-AGENT_TIMEOUT_METADATA_KEY)

[`ENVIRONMENT_SERVER_IMPL`](#nemo_gym-data-tasks-harbor-materialize-ENVIRONMENT_SERVER_IMPL)

[`ENVIRONMENT_SERVER_INSTANCE`](#nemo_gym-data-tasks-harbor-materialize-ENVIRONMENT_SERVER_INSTANCE)

[`STEP_METADATA_KEY`](#nemo_gym-data-tasks-harbor-materialize-STEP_METADATA_KEY)

[`TASK_SOURCE_KEY`](#nemo_gym-data-tasks-harbor-materialize-TASK_SOURCE_KEY)

### API

```python
class nemo_gym.data.tasks.harbor.materialize.HarborTaskInput()
```

**Bases:** `BaseModel`

A Harbor row's input: one `responses_create_params` for a single-turn task, or one per turn.

`task_data` carries the folder digest and the task's own fields; the server reads everything else
from the pinned folder.

**`model_config`** `= ConfigDict(extra='forbid')`

---

**`responses_create_params`** `NeMoGymResponseCreateParamsNonStreaming | None = None`

---

**`task_data`** `dict[str, JsonValue]`

---

**`turns`** `list[HarborTurnInput] = Field(default_factory=list)`

---

```python
nemo_gym.data.tasks.harbor.materialize.HarborTaskInput.accept_flat_task_input(
    value: object
) -> object
```

classmethod

Fold flat task fields into `task_data`, keeping both request-envelope fields where they are.

The flat single-turn input (the task fields beside `responses_create_params`, as
`SingleAgentTurnTaskInput` accepts them) is the canonical row once validated, so a client that sends
`{"responses_create_params": ..., "_ng_digest": ...}` runs as the row does. Only the keys the input set
are passed on, so a row that names no `responses_create_params` (a multi-step row) keeps the field unset
and its serialized form (`exclude_unset`) carries no `null`.

```python
nemo_gym.data.tasks.harbor.materialize.HarborTaskInput.one_shape() -> typing_extensions.Self
```

```python
class nemo_gym.data.tasks.harbor.materialize.HarborTurnInput()
```

**Bases:** `BaseModel`

One turn of a multi-step task: the Harbor step's name and the request the agent receives.

**`model_config`** `= ConfigDict(extra='forbid')`

---

**`name`** `str = Field(min_length=1)`

---

**`responses_create_params`** `NeMoGymResponseCreateParamsNonStreaming`

---

```python
nemo_gym.data.tasks.harbor.materialize._instance_taskset(
    taskset: str
) -> str
```

```python
nemo_gym.data.tasks.harbor.materialize._turn_params(
    instruction: str,
    timeout_sec: float,
    step: str | None = None
) -> dict[str, typing.Any]
```

```python
nemo_gym.data.tasks.harbor.materialize._validate_task_ids(
    tasks: list[nemo_gym.data.tasks.harbor.task.HarborTask]
) -> None
```

```python
nemo_gym.data.tasks.harbor.materialize.agent_instance(
    taskset: str
) -> str
```

Return an override-compatible agent instance name for the taskset.

```python
nemo_gym.data.tasks.harbor.materialize.dataset_entry(
    taskset: str,
    rows_path: pathlib.Path,
    dataset: nemo_gym.data.tasks.harbor.dataset_config.GymDatasetConfig | None = None
) -> dict[str, typing.Any]
```

The `datasets:` entry that declares a taskset's rows, from the dataset's `[gym]` table.

`type` is the split the rows enter (`GymDatasetConfig.effective_type`: the declared one, else
`validation` with a license, else `benchmark`) and `license` the declared one, so a hub dataset
without a license is a benchmark that evaluates and never trains, and a `train` or `validation`
dataset carries the license Gym requires of one. `verified` is the dataset's declaration (`false`
unless it says otherwise, as the pre-commit hook starts a new resources server), and `domain` and
`allowed_agents` are carried when declared. The entry sets no `num_repeats`: collate would duplicate
the rows by it and the collector's run-level count would multiply on top, and the metrics sidecar beside
the rows records the entry, so a count that moved with the command line would conflict run to run.
`[gym.defaults].num_repeats` is the run config's top-level `num_repeats` instead (see
`nemo_gym.data.tasks.harbor.prepare.build_run`). A benchmark entry names no `prepare_script`: the rows are
written by the preparation that generates this entry.

```python
nemo_gym.data.tasks.harbor.materialize.environment_server_instance(
    taskset: str
) -> str
```

Return an override-compatible environment instance name for the taskset.

```python
nemo_gym.data.tasks.harbor.materialize.materialize_task(
    task: nemo_gym.data.tasks.harbor.task.HarborTask,
    taskset: str,
    dataset_digest: str | None = None
) -> nemo_gym.core.episode_types.MaterializedTask[nemo_gym.data.tasks.harbor.materialize.HarborTaskInput]
```

One row: the instruction as a user message, the folder digest and the task's fields in `task_data`.

A multi-step task gets one turn per step, each with that step's instruction and agent budget. The
top-level `instruction.md` is never sent for such a task.

`task_data` holds the task's own field values (`task_data.json` in its folder, validated against
the dataset's `task_data.py` before any row is written) under the reserved `_ng_*` keys. The values
are the file's as written, not the validated model's dump: the model validates strictly, as JSON, so an
accepted value already has its field's type (`validate`),
and the same fields are what the seed compares with the folder's file and what the in-sandbox verifier
reads as `task_data.json`. A dump would drift from both (defaults filled in, aliases and `AliasPath`
shapes rewritten) without changing what any verifier sees.

With `dataset_digest`, the row also carries the dataset's digest (`manifest.toml`'s
`[dataset].digest`). Rollout collection copies the `_ng_*` keys of `task_data` onto every
results row it writes, so each row then says which task folder and which dataset it ran.

```python
nemo_gym.data.tasks.harbor.materialize.rows_metrics_sidecars(
    rows_path: pathlib.Path
) -> tuple[pathlib.Path, pathlib.Path]
```

The data processor's per-dataset metrics beside the rows and the conflict file it writes on a mismatch:
`<rows stem>_metrics.json` and `<rows stem>_metrics_conflict.json` (`nemo_gym.data.dataset_preparation`).

```python
nemo_gym.data.tasks.harbor.materialize.run_config(
    taskset: str,
    folder: pathlib.Path,
    tasks: list[nemo_gym.data.tasks.harbor.task.HarborTask],
    rows_path: pathlib.Path,
    agent_instance_name: str,
    agent_impl: str,
    agent_overrides: dict[str, typing.Any] | None = None,
    sandbox_provider: str = 'sandbox',
    dataset_digest: str | None = None,
    dataset_toml_digest: str | None = None,
    dataset: nemo_gym.data.tasks.harbor.dataset_config.GymDatasetConfig | None = None
) -> dict[str, typing.Any]
```

The run-config blocks that bind one taskset to the `harbor` environment server.

The environment server block inherits the shipped `harbor_environment_server` config and
names the taskset's agent. The agent block inherits the selected harness config and declares
the taskset's rows as its dataset (`dataset_entry`, from the `[gym]` table of
`dataset.toml`: split, license, repeats and catalog metadata), which is where rollout collection
looks for datasets. Rows route to the environment server by taskset. The taskset mapping records the
dataset digest and the `dataset.toml` digest, so the run config says which dataset ran and
whether the Gym layer or upstream moved between two runs.

```python
nemo_gym.data.tasks.harbor.materialize.taskset_mapping(
    tasks: list[nemo_gym.data.tasks.harbor.task.HarborTask],
    folder: pathlib.Path
) -> dict[str, typing.Any]
```

Where a taskset's folders live and which digest each task had at materialization.

Servers resolve a task as `<folder>/<task_id>`, or `<folder>/<paths[task_id]>` for a task
found below the first level (see `nemo_gym.data.tasks.harbor.task.discover_tasks`). For a
folder of task folders `folder` is the folder itself; for a single task folder it is the
task's parent, so `folder` is always the root the tasks were discovered under.

```python
nemo_gym.data.tasks.harbor.materialize.write_rows(
    tasks: list[nemo_gym.data.tasks.harbor.task.HarborTask],
    taskset: str,
    path: pathlib.Path,
    dataset_digest: str | None = None,
    task_source: str | None = None
) -> list[dict[str, typing.Any]]
```

Write one materialized row per task as JSONL and return the rows.

Beside the task's identity and input, every row carries `task_source`: the dataset folder's name,
`task_source` when given (a single task folder target inside a dataset passes its parent's name; a
standalone task folder, whose parent holds no `dataset.toml` or `manifest.toml`, passes its own) and
the taskset otherwise (a dataset folder target's taskset is that name). It is the dataset label a results row and
a trainer read, and nothing the collector routes by: a materialized row reaches its environment server
by its taskset, `agent_map` and `fan_out` do not apply to it, and a per-dataset `num_repeats` dict
keys on the taskset.

The metrics sidecars the data processor leaves beside the rows (`rows_metrics_sidecars`) go with the
rows they described: they are derived from one selection of the task folder, and the processor would refuse
a changed selection against them ("Found conflicting aggregate metrics"), a check meant for a committed
source dataset, not for rows regenerated on every command. Removed here, they are rewritten by the run.

```python
nemo_gym.data.tasks.harbor.materialize.AGENT_TIMEOUT_METADATA_KEY = 'agent_timeout_sec'
```

```python
nemo_gym.data.tasks.harbor.materialize.ENVIRONMENT_SERVER_IMPL = 'harbor'
```

```python
nemo_gym.data.tasks.harbor.materialize.ENVIRONMENT_SERVER_INSTANCE = 'harbor_environment_server'
```

```python
nemo_gym.data.tasks.harbor.materialize.STEP_METADATA_KEY = 'harbor_step'
```

```python
nemo_gym.data.tasks.harbor.materialize.TASK_SOURCE_KEY = 'task_source'
```