nemo_gym.tasks.harbor.image_configs

The task’s Compose overlay, if it has one.

Every image a Compose task starts: the task image for main and each sidecar’s image.

The base image of every task Dockerfile the loader resolves under folder, with the tasks using it.

folder is a task folder or a folder of task folders, as discover_tasks reads it; with select, only the tasks whose folder name it accepts count. The image a task declares is the effective one, exactly as load_task sees it: task.toml after the dataset’s [gym] overrides. A per-task docker_image override thus turns a Dockerfile with RUN lines into a prebuilt image whose FROM is never resolved, so there is nothing to record for it. Tasks whose task.toml does not load, and a dataset whose dataset.toml does not load (which fails every task), are left for the loader to report.

The task’s Compose overlay with env_file entries inlined and ${VAR} references resolved.

Returns None for a task without a Compose file. Prepare and the resources server both read the file through this function, so the document the sandbox starts is the one prepare validated, provided both run with the same process environment (see compose_variables). A required variable (${VAR:?msg}) without a value, a task variable the host does not set or a missing env_file raises ImageConfigError naming it; a variable nobody sets is an empty string, with a warning.

Record the base image of every Dockerfile under folder next to the task folders, before they load.

select limits the scan to the tasks whose folder name it accepts (see dockerfile_base_images). Returns the file’s path, or None when no task has a Dockerfile the loader resolves.

Write compose-images.json in folder for the Compose tasks among tasks.

Returns the file’s path, or None when no task uses Compose. A task whose Compose file does not resolve (an ImageConfigError from load_compose_document) does not stop the others: with skipped given it is recorded there by task id with the reason and left out, otherwise one ImageConfigError at the end names every such task. An image the registry cannot resolve still raises for the whole set, as record_image_configs says.

Record the configuration of references (image reference to the tasks using it) into folder.

Entries already recorded are kept as they are, so a pinned recording never changes under a completed run; only references missing from the file are resolved. An entry an older Gym wrote (no current record_version: it lacks fields the Dockerfile resolver reads) is read again, at the digest it was pinned to rather than the tag, so the pin does not move either. Returns the file’s path, or None when there is nothing to record. A reference the registry cannot resolve raises ImageConfigError naming the image, the tasks and the registry’s answer.

The registry is asked outside the file’s sidecar lock (record_lock), so a slow registry never holds up another process; the file is then read again, merged and replaced under the lock, and an entry another process recorded in the meantime is kept over the one resolved here, as a pinned entry always is (a legacy entry is replaced by its re-read).

document with main’s build and pull_policy removed, for a task that runs from the image prepare built and recorded.

Harbor builds main from the task’s Compose file at every start, so the file may give main a build: (args, a target, a dockerfile_inline) and Harbor’s own template sets pull_policy: build. Gym builds that image once at prepare, with the same file, and the loader then substitutes the recorded hb__<hash> image for main (MAIN_IMAGE_NAME resolves to it). The two keys describe a build that already happened; the sandbox Compose adapter starts images only and refuses them as unsupported fields, so they are dropped here rather than failing every episode of a task prepare accepted. Nothing else is touched: a sidecar’s build: is refused at prepare, and a task in pull or overlay mode goes through without_restated_main_build instead.

document with pull_policy removed from every service, for the sandbox Compose adapter.

The key tells docker compose when to pull or build an image before starting it (Harbor’s template sets pull_policy: build on main; a task file may give a sidecar always or missing). The sandbox starts the recorded image it is given and has no pull step of its own to configure, so the key is dropped from the document the sandbox receives, in every image mode, instead of being refused as an unsupported field. build: is not touched here: a sidecar’s is still refused (no provider starts a locally built sidecar) and main’s is dropped by without_main_build for a task that runs from its recorded built image, or by without_restated_main_build when it only restates Harbor’s build.

document with main’s build: dropped when it only restates Harbor’s own build of the task’s Dockerfile, for a task in pull or overlay mode, and a reason instead when that section carries a key Gym cannot apply without a build. Any other task is returned as it is.

Harbor builds main from environment/Dockerfile through its own template, which already sets build.context: ${CONTEXT_DIR} (docker-compose-build.yaml at v0.22.0, appended before the task file), so a task file that repeats context, dockerfile: Dockerfile or pull changes nothing there; orca-bench, terminal_bench_aai and data-eng-bench do, some 600 hub tasks. Gym runs such a Dockerfile without a build (the loader folded it into the task in pull mode, or runs its RUN lines at seed in overlay mode), so the restatement is dropped as a built task’s section is (without_main_build). args, target, dockerfile_inline, another dockerfile or another context would change the image Harbor builds and have no equivalent without one, so the task is reported, not started on the wrong image.