> This page is for version Main (default).
> For other versions, use one of these documentation indexes:
> - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt
> - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt
> - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt
> - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt
> - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt
> - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt
> - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt
> - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# nemo_gym.data.tasks.harbor.python_verifier

Python verifiers: which module grades a task, and the `test.sh` shim that runs it inside the sandbox.

A task may grade with `tests/verifier.py` instead of `tests/test.sh`; a dataset may ship one `verifier.py`
beside `dataset.toml` that grades every task without a `tests/` folder. `tests/test.sh` always wins when it is
present: many hub datasets ship a `tests/verifier.py` that their `test.sh` calls as a helper, and such a task is a
plain `test.sh` task to Gym (the module is uploaded with `tests/` like any other file). The module's `verify`
function (see `nemo_gym.verifiers`) never runs on the host. The `harbor` environment server uploads a shim folder
over `/tests` after the task's own `tests/`: a `test.sh` that finds a Python (3.8 or newer, `PYTHON_FLOOR`;
an older one is refused with the version named, exit 127, no reward), the runner, a vendored copy
of `nemo_gym.verifiers`, the task's fields and, at run time, the agent's final response, and runs the
verifier with the workspace path. `gym dataset publish --harbor` writes the same shim into each expanded
task's `tests/` (without the response, which Harbor cannot provide): the task's own `tests/verifier.py` stays
where it is and the shim runs it there, a dataset-level module travels under `tests/.ng/verifier.py`, so the
published folder grades by `tests/test.sh` the same way under Harbor in shared verifier mode, and loads back into
Gym as a plain `test.sh` task. Publish refuses a Python
verifier in separate mode: Harbor uploads nothing into a separate verifier container, so the shim could not reach it.

The shim clears any `reward.json` or `reward.txt` already in `/logs/verifier` before the runner starts, and the
runner does the same before `verify` runs. Harbor does not: in shared mode its trial bind-mounts the verifier log
folder into the agent's container for the whole run and reads `reward.json` before `reward.txt`, so a file the agent
planted there would otherwise stand as the verdict (beside, or instead of, the runner's own). Gym's server empties the
folder before every upload; the published artifact is graded by Harbor, which is why the shim clears it itself.

## Module Contents

### Classes

| Name                                                                                     | Description                                                             |
| ---------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
| [`PythonVerifierError`](#nemo_gym-data-tasks-harbor-python_verifier-PythonVerifierError) | A task's Python verifier is not usable (a module the shim cannot wrap). |
| [`VerifierShim`](#nemo_gym-data-tasks-harbor-python_verifier-VerifierShim)               | What the shim needs to know about the task it wraps.                    |

### Functions

| Name                                                                                     | Description                                                                                         |
| ---------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
| [`declared_arguments`](#nemo_gym-data-tasks-harbor-python_verifier-declared_arguments)   | The parameter names of `def verify` in the module, read from its source without importing it.       |
| [`python_verifier_for`](#nemo_gym-data-tasks-harbor-python_verifier-python_verifier_for) | The module that grades `task_folder`, or `None` when `tests/test.sh` does.                          |
| [`reads_response`](#nemo_gym-data-tasks-harbor-python_verifier-reads_response)           | Whether `verify` declares `response`: such a verifier cannot be published to Harbor.                |
| [`test_script`](#nemo_gym-data-tasks-harbor-python_verifier-test_script)                 | The `test.sh` the sandbox runs: clear stale reward files, find a Python of at least `PYTHON_FLOOR`, |
| [`write_shim`](#nemo_gym-data-tasks-harbor-python_verifier-write_shim)                   | Write the shim folder at `target` (created; existing shim files replaced) and return it.            |

### Data

[`PYTHON_FLOOR`](#nemo_gym-data-tasks-harbor-python_verifier-PYTHON_FLOOR)

[`RESPONSE_FILE`](#nemo_gym-data-tasks-harbor-python_verifier-RESPONSE_FILE)

[`REWARD_DIR`](#nemo_gym-data-tasks-harbor-python_verifier-REWARD_DIR)

[`RUNNER_FILE`](#nemo_gym-data-tasks-harbor-python_verifier-RUNNER_FILE)

[`SHIM_DIR`](#nemo_gym-data-tasks-harbor-python_verifier-SHIM_DIR)

[`TASK_DATA_FILE`](#nemo_gym-data-tasks-harbor-python_verifier-TASK_DATA_FILE)

[`TESTS_DIR`](#nemo_gym-data-tasks-harbor-python_verifier-TESTS_DIR)

[`TEST_SCRIPT`](#nemo_gym-data-tasks-harbor-python_verifier-TEST_SCRIPT)

[`VERIFIER_FILE`](#nemo_gym-data-tasks-harbor-python_verifier-VERIFIER_FILE)

[`logger`](#nemo_gym-data-tasks-harbor-python_verifier-logger)

### API

```python
class nemo_gym.data.tasks.harbor.python_verifier.PythonVerifierError()
```

**Bases:** [HarborTaskError](/nemo/gym/nemo-gym/nemo_gym/data/tasks/harbor/errors#nemo_gym-data-tasks-harbor-errors-HarborTaskError)

A task's Python verifier is not usable (a module the shim cannot wrap).

```python
class nemo_gym.data.tasks.harbor.python_verifier.VerifierShim(
    verifier: pathlib.Path,
    in_task_tests: bool,
    workspace: str | None,
    task_data: typing.Mapping[str, typing.Any],
    response: typing.Mapping[str, typing.Any] | None = None
)
```

Dataclass

What the shim needs to know about the task it wraps.

**`in_task_tests`** `bool`

---

**`remote_verifier`** `str`

---

**`response`** `Mapping[str, Any] | None = None`

---

**`task_data`** `Mapping[str, Any]`

---

**`verifier`** `Path`

---

**`workspace`** `str | None`

---

```python
nemo_gym.data.tasks.harbor.python_verifier.declared_arguments(
    module_path: pathlib.Path
) -> frozenset[str] | None
```

The parameter names of `def verify` in the module, read from its source without importing it.

`None` when the module defines `verify` some other way (`verify = partial(...)`), which the host cannot
resolve without running task code; the in-sandbox runner then decides at run time.

```python
nemo_gym.data.tasks.harbor.python_verifier.python_verifier_for(
    task_folder: pathlib.Path,
    dataset_root: pathlib.Path
) -> pathlib.Path | None
```

The module that grades `task_folder`, or `None` when `tests/test.sh` does.

`tests/verifier.py` when the task has one and no `tests/test.sh`; else the dataset root's `verifier.py`
when the task has no `tests/` folder at all. `tests/test.sh` wins whenever it exists: on the hub a
`tests/verifier.py` next to it is the helper that `test.sh` runs (the shape of whole datasets, such as
multimedia-terminalbench, osworld-verified or o11y-bench), not a second grader, so the task is a `test.sh` task
and the module an ordinary test file. A `tests/` folder holding neither is left to `test.sh` grading
(`None`): Harbor admits that shape when a separate verifier image ships its own `/tests/test.sh`, and the
loader kept admitting it; such a task without one fails at verification with no `test.sh` to run, as before.

```python
nemo_gym.data.tasks.harbor.python_verifier.reads_response(
    module_path: pathlib.Path
) -> bool
```

Whether `verify` declares `response`: such a verifier cannot be published to Harbor.

```python
nemo_gym.data.tasks.harbor.python_verifier.test_script(
    shim: nemo_gym.data.tasks.harbor.python_verifier.VerifierShim
) -> str
```

The `test.sh` the sandbox runs: clear stale reward files, find a Python of at least `PYTHON_FLOOR`,
hand the runner the paths, let it write the reward. No Python, or an older one, is exit 127 with the reason on
stderr and no reward (the missing-reward outcome).

```python
nemo_gym.data.tasks.harbor.python_verifier.write_shim(
    target: pathlib.Path,
    shim: nemo_gym.data.tasks.harbor.python_verifier.VerifierShim
) -> pathlib.Path
```

Write the shim folder at `target` (created; existing shim files replaced) and return it.

`target/test.sh`, `target/.ng/run_verifier.py`, `target/.ng/nemo_gym/verifiers/*` (the installed
package's modules, with an empty `nemo_gym/__init__.py` so the sandbox never imports the real package),
`target/.ng/task_data.json`, `target/.ng/response.json` when a response is given, and
`target/.ng/verifier.py` when the module is not in the task's own `tests/`.

The generated tree is world-readable (`0755` folders, `0644` files, `test.sh` `0755`) whatever umask the
process runs under. Gym's upload and Harbor's `docker cp` both keep the archive's modes and chown nothing, and
`test.sh` runs as the task's `[verifier].user`: under `umask 077` the runner and its helpers would land
`0600` in a `0700` folder that a non-root verifier user could not open, and grading would fail before
`verify` ran. Only the shim's own files are set, and `target` itself when this call created it (publish's
`tests/` for a task graded by the dataset's module, which Harbor copies into `/tests` modes and all); a
`tests/` the task shipped keeps the modes its author gave it.

```python
nemo_gym.data.tasks.harbor.python_verifier.PYTHON_FLOOR = (3, 8)
```

```python
nemo_gym.data.tasks.harbor.python_verifier.RESPONSE_FILE = 'response.json'
```

```python
nemo_gym.data.tasks.harbor.python_verifier.REWARD_DIR = '/logs/verifier'
```

```python
nemo_gym.data.tasks.harbor.python_verifier.RUNNER_FILE = 'run_verifier.py'
```

```python
nemo_gym.data.tasks.harbor.python_verifier.SHIM_DIR = '.ng'
```

```python
nemo_gym.data.tasks.harbor.python_verifier.TASK_DATA_FILE = 'task_data.json'
```

```python
nemo_gym.data.tasks.harbor.python_verifier.TESTS_DIR = '/tests'
```

```python
nemo_gym.data.tasks.harbor.python_verifier.TEST_SCRIPT = 'test.sh'
```

```python
nemo_gym.data.tasks.harbor.python_verifier.VERIFIER_FILE = 'verifier.py'
```

```python
nemo_gym.data.tasks.harbor.python_verifier.logger = logging.getLogger(__name__)
```