> This page is for version 0.7.0.
> For other versions, use one of these documentation indexes:
> - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt
> - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt
> - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt
> - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt
> - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt
> - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt
> - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt
> - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# nemo_gym.tasks.harbor.validation_report

Summarise a gym dataset validate run from its rollout and failure files.

Reads the rows a validation run wrote, keys them by task, reports which tasks passed, which
scored below one and which were never run, and returns the text and the overall verdict as a
ValidationReport. The task digest lookup the resume checks share lives here as well. cli.py
imports it for validate_target and the eval run resume conflict check.

## Module Contents

### Classes

| Name                                                                            | Description |
| ------------------------------------------------------------------------------- | ----------- |
| [`ValidationReport`](#nemo_gym-tasks-harbor-validation_report-ValidationReport) | -           |

### Functions

| Name                                                                                    | Description                                                                                             |
| --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- |
| [`_dig`](#nemo_gym-tasks-harbor-validation_report-_dig)                                 | -                                                                                                       |
| [`_task_digests`](#nemo_gym-tasks-harbor-validation_report-_task_digests)               | `key(row) -> task digest` for materialized rows; shared by the eval and validation resume checks.    |
| [`_task_name`](#nemo_gym-tasks-harbor-validation_report-_task_name)                     | -                                                                                                       |
| [`_validation_key`](#nemo_gym-tasks-harbor-validation_report-_validation_key)           | -                                                                                                       |
| [`_validation_rows`](#nemo_gym-tasks-harbor-validation_report-_validation_rows)         | The records of a rollouts, failures or materialized inputs file, in order (none when there is no file). |
| [`summarize_validation`](#nemo_gym-tasks-harbor-validation_report-summarize_validation) | Report every selected task/repeat, reconciling results with failed attempts.                            |

### API

```python
class nemo_gym.tasks.harbor.validation_report.ValidationReport(
    text: str,
    ok: bool
)
```

Dataclass

**`ok`** `bool`

---

**`text`** `str`

---

```python
nemo_gym.tasks.harbor.validation_report._dig(
    mapping: typing.Any,
    keys: str = ()
) -> typing.Any
```

```python
nemo_gym.tasks.harbor.validation_report._task_digests(
    rows: list[dict[str, object]],
    key: typing.Any
) -> dict[typing.Any, object]
```

`key(row) -> task digest` for materialized rows; shared by the eval and validation resume checks.

```python
nemo_gym.tasks.harbor.validation_report._task_name(
    row: dict[str, typing.Any]
) -> str
```

```python
nemo_gym.tasks.harbor.validation_report._validation_key(
    row: dict[str, object]
) -> tuple[str, str, object, object]
```

```python
nemo_gym.tasks.harbor.validation_report._validation_rows(
    path: pathlib.Path
) -> list[dict[str, object]]
```

The records of a rollouts, failures or materialized inputs file, in order (none when there is no file).

A run killed while appending leaves a final line without a newline that does not parse (the cut may
land inside a multibyte character, so it may not decode either). The collector drops that line from
the rollouts and failures files when it resumes (`_drop_truncated_tail`), so it is skipped here with
the same tolerance, and a resume that reads the cache first still reaches that repair. A complete final
record without a newline is read like any other. A line that does not parse anywhere else is damage
the collector does not repair, and is reported by file and line rather than resumed over.

```python
nemo_gym.tasks.harbor.validation_report.summarize_validation(
    rollouts: pathlib.Path,
    unvalidated: list[str],
    expected: list[dict[str, object]] | None = None,
    skipped: dict[str, str] | None = None
) -> nemo_gym.tasks.harbor.validation_report.ValidationReport
```

Report every selected task/repeat, reconciling results with failed attempts.

The collector stores verify responses directly and keeps historical failed attempts in a
sidecar. A result supersedes failures for the same task/repeat, but never another repeat.
`expected` contains the collector's selected, expanded input rows (after limit/repeats).