> This page is for version 0.5.0.
> For other versions, use one of these documentation indexes:
> - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt
> - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt
> - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt
> - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt
> - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt
> - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt
> - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt
> - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# nemo_gym.data.tasks.harbor.validation_report

Summarise a gym dataset validate run from its rollout and failure files.

Reads the rows a validation run wrote, keys them by task, reports which tasks passed, which
scored below one and which were never run, and returns the text and the overall verdict as a
ValidationReport. The task digest lookup the resume checks share lives here as well.
`nemo_gym.cli.eval` imports it for `validate_target` and for the resume conflict check of `gym eval run`.

## Module Contents

### Classes

| Name                                                                                 | Description |
| ------------------------------------------------------------------------------------ | ----------- |
| [`ValidationReport`](#nemo_gym-data-tasks-harbor-validation_report-ValidationReport) | -           |

### Functions

| Name                                                                                           | Description                                                                                             |
| ---------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- |
| [`_dig`](#nemo_gym-data-tasks-harbor-validation_report-_dig)                                   | -                                                                                                       |
| [`_iter_validation_rows`](#nemo_gym-data-tasks-harbor-validation_report-_iter_validation_rows) | The records of a rollouts, failures or materialized inputs file, in order (none when there is no file). |
| [`_task_digests`](#nemo_gym-data-tasks-harbor-validation_report-_task_digests)                 | `key(row) -> task digest` for materialized rows; shared by the eval and validation resume checks.    |
| [`_task_name`](#nemo_gym-data-tasks-harbor-validation_report-_task_name)                       | -                                                                                                       |
| [`_validation_key`](#nemo_gym-data-tasks-harbor-validation_report-_validation_key)             | -                                                                                                       |
| [`_validation_rows`](#nemo_gym-data-tasks-harbor-validation_report-_validation_rows)           | `_iter_validation_rows` as a list, for the readers that need every record at once.                      |
| [`summarize_validation`](#nemo_gym-data-tasks-harbor-validation_report-summarize_validation)   | Report every selected task/repeat, reconciling results with failed attempts.                            |

### API

```python
class nemo_gym.data.tasks.harbor.validation_report.ValidationReport(
    text: str,
    ok: bool
)
```

Dataclass

**`ok`** `bool`

---

**`text`** `str`

---

```python
nemo_gym.data.tasks.harbor.validation_report._dig(
    mapping: typing.Any,
    keys: str = ()
) -> typing.Any
```

```python
nemo_gym.data.tasks.harbor.validation_report._iter_validation_rows(
    path: pathlib.Path
) -> collections.abc.Iterator[dict[str, object]]
```

The records of a rollouts, failures or materialized inputs file, in order (none when there is no file).

One record is read and parsed at a time, so a reader that keeps a projection of each row (the run
summary) never holds a long run's responses together. A run killed while appending leaves a final line
without a newline that does not parse (the cut may land inside a multibyte character, so it may not
decode either). The collector drops that line from the rollouts and failures files when it resumes
(`_drop_truncated_tail`), so it is skipped here with the same tolerance, and a resume that reads the
cache first still reaches that repair. A complete final record without a newline is read like any
other. A line that does not parse anywhere else is damage the collector does not repair, and is reported
by file and line rather than resumed over.

```python
nemo_gym.data.tasks.harbor.validation_report._task_digests(
    rows: list[dict[str, object]],
    key: typing.Any
) -> dict[typing.Any, object]
```

`key(row) -> task digest` for materialized rows; shared by the eval and validation resume checks.

```python
nemo_gym.data.tasks.harbor.validation_report._task_name(
    row: dict[str, typing.Any]
) -> str
```

```python
nemo_gym.data.tasks.harbor.validation_report._validation_key(
    row: dict[str, object]
) -> tuple[str, str, object, object]
```

```python
nemo_gym.data.tasks.harbor.validation_report._validation_rows(
    path: pathlib.Path
) -> list[dict[str, object]]
```

`_iter_validation_rows` as a list, for the readers that need every record at once.

```python
nemo_gym.data.tasks.harbor.validation_report.summarize_validation(
    rollouts: pathlib.Path,
    unvalidated: list[str],
    expected: list[dict[str, object]] | None = None,
    skipped: dict[str, str] | None = None
) -> nemo_gym.data.tasks.harbor.validation_report.ValidationReport
```

Report every selected task/repeat, reconciling results with failed attempts.

The collector stores verify responses directly and keeps historical failed attempts in a
sidecar. A result supersedes failures for the same task/repeat, but never another repeat.
`expected` contains the collector's selected, expanded input rows (after limit/repeats).