> This page is for version 0.7.0.
> For other versions, use one of these documentation indexes:
> - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt
> - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt
> - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt
> - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt
> - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt
> - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt
> - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt
> - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# nemo_gym.data.tasks.harbor.digest

Whole-folder digest of a Harbor task.

Same record format as Harbor's package publisher and Gym's Terminal Bench 4 loader:
SHA256 over sorted `relative_path NUL file_sha256 LF` records covering
`task.toml`, `instruction.md`, `README.md`, `trajectory.json` and everything
under `environment/`, `tests/`, `solution/` and `steps/`. The digest travels in
`task_data` under `DIGEST_KEY` so the resources server can refuse a row whose
folder changed after materialization. `content_hash` refuses directory symlinks because
this file-content format does not represent their targets.

Beside it, `DATASET_DIGEST_KEY` carries the digest of
the whole dataset (`manifest.toml`'s `[dataset].digest`, see `nemo_gym.data.tasks.harbor.manifest`). Rollout
collection copies both `_ng_*` keys from the row onto the results row it writes, so every
`rollouts.jsonl` row says which task folder and which dataset it ran. `context_digest` is the second
digest here: Harbor's environment id of a Dockerfile build context, which keys and names the images prepare
builds. `inputs_digest` is the third: everything a build of that context reads, recorded with the built
image so a context whose Harbor id is unchanged but whose inputs are not (a mode, a symbolic link, an empty
folder, a build setting) is rebuilt, and `inputs_differences` spells out where two trees of one Harbor id
part, for the task left out of a shared build.

## Module Contents

### Functions

| Name                                                                          | Description                                                                                               |
| ----------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
| [`_covered_files`](#nemo_gym-data-tasks-harbor-digest-_covered_files)         | The regular files under the resolved task folder `path` that a digest covers, in record order.            |
| [`_covered_paths`](#nemo_gym-data-tasks-harbor-digest-_covered_paths)         | Every entry under the resolved task folder `path` that a digest reads, links included: the top-level task |
| [`_describe_entry`](#nemo_gym-data-tasks-harbor-digest-_describe_entry)       | -                                                                                                         |
| [`_record`](#nemo_gym-data-tasks-harbor-digest-_record)                       | One `relative_path NUL file_sha256 LF` record of `file` under the resolved task folder `path`.            |
| [`_tree_entries`](#nemo_gym-data-tasks-harbor-digest-_tree_entries)           | Every entry under the context folder `context` as `inputs_digest` reads it, sorted by relative posix      |
| [`archive_hash`](#nemo_gym-data-tasks-harbor-digest-archive_hash)             | Harbor's published content hash of the unpacked task package at `path`.                                   |
| [`content_hash`](#nemo_gym-data-tasks-harbor-digest-content_hash)             | Digest the task folder at `path`.                                                                         |
| [`context_digest`](#nemo_gym-data-tasks-harbor-digest-context_digest)         | Harbor's environment id of a Dockerfile build context (a task's `environment/` folder).                   |
| [`inputs_differences`](#nemo_gym-data-tasks-harbor-digest-inputs_differences) | Where the context folders `ours` and `theirs` differ in what `inputs_digest` reads, in path order.        |
| [`inputs_digest`](#nemo_gym-data-tasks-harbor-digest-inputs_digest)           | Digest of everything an image build of the context folder at `context` reads.                             |
| [`is_covered`](#nemo_gym-data-tasks-harbor-digest-is_covered)                 | Whether the task path at `relative` (a posix path under the task folder) is one `content_hash`            |
| [`is_ignored`](#nemo_gym-data-tasks-harbor-digest-is_ignored)                 | Whether `content_hash` skips the task file at `relative` (a posix path under the task folder):            |

### Data

[`DATASET_DIGEST_KEY`](#nemo_gym-data-tasks-harbor-digest-DATASET_DIGEST_KEY)

[`DIGEST_KEY`](#nemo_gym-data-tasks-harbor-digest-DIGEST_KEY)

[`GITIGNORE`](#nemo_gym-data-tasks-harbor-digest-GITIGNORE)

[`_DIRECTORIES`](#nemo_gym-data-tasks-harbor-digest-_DIRECTORIES)

[`_IGNORED`](#nemo_gym-data-tasks-harbor-digest-_IGNORED)

[`_KIND_NAMES`](#nemo_gym-data-tasks-harbor-digest-_KIND_NAMES)

[`_SINGLE_FILES`](#nemo_gym-data-tasks-harbor-digest-_SINGLE_FILES)

### API

```python
nemo_gym.data.tasks.harbor.digest._covered_files(
    path: pathlib.Path
) -> list[pathlib.Path]
```

The regular files under the resolved task folder `path` that a digest covers, in record order.

```python
nemo_gym.data.tasks.harbor.digest._covered_paths(
    path: pathlib.Path
) -> list[pathlib.Path]
```

Every entry under the resolved task folder `path` that a digest reads, links included: the top-level task
files and the covered folders with everything below them. Entries outside them (an author's `.venv/` or
`.git/`) are not the task's and never reach a digest.

```python
nemo_gym.data.tasks.harbor.digest._describe_entry(
    entry: tuple[str, int, str] | None,
    against: tuple[str, int, str] | None
) -> str | None
```

```python
nemo_gym.data.tasks.harbor.digest._record(
    path: pathlib.Path,
    file: pathlib.Path
) -> bytes
```

One `relative_path NUL file_sha256 LF` record of `file` under the resolved task folder `path`.

```python
nemo_gym.data.tasks.harbor.digest._tree_entries(
    context: pathlib.Path
) -> list[tuple[str, str, int, str]]
```

Every entry under the context folder `context` as `inputs_digest` reads it, sorted by relative posix
path: `(relative_path, kind, st_mode, payload)`.

```python
nemo_gym.data.tasks.harbor.digest.archive_hash(
    path: pathlib.Path
) -> str
```

Harbor's published content hash of the unpacked task package at `path`.

The same records as `content_hash` over every covered file, with no exclusions: Harbor's
packager applies the task's `.gitignore` (or its defaults, the same names as `_IGNORED`)
when it chooses the archive's files, and the published hash covers exactly those files. A
downloaded archive therefore matches its advertised hash by this digest, whatever the publisher
excluded or kept, while `content_hash` stays the digest of a task folder on disk, where
byte-code caches and editor leftovers appear after a run and must not change the task's identity.

```python
nemo_gym.data.tasks.harbor.digest.content_hash(
    path: pathlib.Path
) -> str
```

Digest the task folder at `path`.

```python
nemo_gym.data.tasks.harbor.digest.context_digest(
    path: pathlib.Path
) -> str
```

Harbor's environment id of a Dockerfile build context (a task's `environment/` folder).

`environment_content_hash` over the folder: 32 hex
characters of SHA-256 over the sorted (relative path, bytes) of its regular files. Symbolic links and
anything under a `_CONTENT_HASH_IGNORE_NAMES` member
(`__pycache__`, `.git`, `.DS_Store`) are skipped, and file modes are not part of it: parity with
Harbor's image name is the contract, not a complete cache identity. An empty folder hashes to its
name, as Harbor hashes it when the task declares no `docker_image` (one that does never reaches a
build). Two contexts with the same files get the same id, so it keys the build record (see
`nemo_gym.data.tasks.harbor.image_build`); whether a recorded image is reused is `inputs_digest`'s call.

```python
nemo_gym.data.tasks.harbor.digest.inputs_differences(
    ours: pathlib.Path,
    theirs: pathlib.Path
) -> list[tuple[str, str | None, str | None]]
```

Where the context folders `ours` and `theirs` differ in what `inputs_digest` reads, in path order.

One item per entry that is in one tree only or reads differently in the two: `(relative_path, in ours, in
theirs)`, each side a short description (`a file 0755`, `a link 0777 -> target`, `a folder 0755`, with
`(content <sha256 prefix>)` on a file whose bytes differ from the other side's) or `None` where the tree has
no such entry. Empty exactly when the two trees' `inputs_digest` agree.

```python
nemo_gym.data.tasks.harbor.digest.inputs_digest(
    context: pathlib.Path,
    settings: collections.abc.Mapping[str, str] | None = None
) -> str
```

Digest of everything an image build of the context folder at `context` reads.

Where `context_digest` is Harbor's name for the context, this is the build's identity: SHA-256 (64 hex
characters) over every entry under `context`, sorted by relative posix path, as
`relative_path NUL kind NUL st_mode NUL payload LF`. `kind` is `f` for a regular file (payload: the
SHA-256 of its bytes), `l` for a symbolic link (payload: its target, read, not followed), `d` for a
folder (no payload, so an empty folder counts) and `o` for anything else; `st_mode` is the full
`lstat` mode in octal, permission bits included. Nothing is skipped: the names Harbor's hash ignores
(`__pycache__`, `.git`, `.DS_Store`) and symbolic links are part of it, since a build tool reads them.
`settings` are the build's inputs outside the tree (the Dockerfile path, the build arguments), appended
as `NUL key NUL value LF` records sorted by key. Folder ownership is not part of it: a build does not
read it, and it differs between hosts that hold the same folder.

```python
nemo_gym.data.tasks.harbor.digest.is_covered(
    relative: str
) -> bool
```

Whether the task path at `relative` (a posix path under the task folder) is one `content_hash`
covers: one of the top-level task files, or anything under `environment/`, `tests/`, `solution/` or
`steps/` (those folders themselves included), and not a name `is_ignored` skips. A folder that is
covered may still hold skipped names after a run; they do not change the task's identity.

```python
nemo_gym.data.tasks.harbor.digest.is_ignored(
    relative: str
) -> bool
```

Whether `content_hash` skips the task file at `relative` (a posix path under the task folder):
editor leftovers and byte-code caches, matched on any path component.

```python
nemo_gym.data.tasks.harbor.digest.DATASET_DIGEST_KEY = '_ng_dataset_digest'
```

```python
nemo_gym.data.tasks.harbor.digest.DIGEST_KEY = '_ng_digest'
```

```python
nemo_gym.data.tasks.harbor.digest.GITIGNORE = '.gitignore'
```

```python
nemo_gym.data.tasks.harbor.digest._DIRECTORIES = ('environment', 'tests', 'solution', 'steps')
```

```python
nemo_gym.data.tasks.harbor.digest._IGNORED = ('__pycache__', '*.pyc', '.DS_Store', '*.swp', '*.swo', '*~')
```

```python
nemo_gym.data.tasks.harbor.digest._KIND_NAMES = {'f': 'a file', 'l': 'a link', 'd': 'a folder', 'o': 'another kind of entry'}
```

```python
nemo_gym.data.tasks.harbor.digest._SINGLE_FILES = ('task.toml', 'instruction.md', 'README.md', 'trajectory.json', 'task_data.json'...
```