> This page is for version 0.5.0. > For other versions, use one of these documentation indexes: > - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt > - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt > - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt > - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt > - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt > - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt > - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt > - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server. # nemo_gym.tasks.harbor.build_tools Drive docker, buildah or podman for one planned build. ImageBuilder owns the per-tool argv for build, compose build, push and inspect, the generated containers.conf and registries.conf environment, the compose plugin and heredoc preflights, the pushed digest and the inspect normalisation that turns tool output into the recorded configuration. It is the only build module that runs subprocesses, through the Runner seam. ImageBuilder resolves detect_tool from this module's globals, which is where tests patch it. ## Module Contents ### Classes | Name | Description | | ----------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | | [`ImageBuilder`](#nemo_gym-tasks-harbor-build_tools-ImageBuilder) | Build one context into a named image with docker, buildah or podman, and push it when asked. | ### Functions | Name | Description | | --------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | | [`_docker_config`](#nemo_gym-tasks-harbor-build_tools-_docker_config) | The set fields of an inspect's `Docker.config` section (the docker-format image configuration), or `{}`. | | [`_proxy_note`](#nemo_gym-tasks-harbor-build_tools-_proxy_note) | Why a registry step may have answered 403, when the tool's output says it did: a proxy the build tool | | [`_pushed_digest`](#nemo_gym-tasks-harbor-build_tools-_pushed_digest) | The manifest digest the registry assigned to the pushed image. | | [`_run_default`](#nemo_gym-tasks-harbor-build_tools-_run_default) | - | | [`normalize_inspect`](#nemo_gym-tasks-harbor-build_tools-normalize_inspect) | Turn a tool's inspect JSON into the `compose-images.json` record shape (without `image`). | | [`proxy_variable_names`](#nemo_gym-tasks-harbor-build_tools-proxy_variable_names) | The proxy-related variables in `env` (`http_proxy`, `HTTPS_PROXY`, `no_proxy`, `all_proxy` and the | ### Data [`BUILT_CONFIG_KEYS`](#nemo_gym-tasks-harbor-build_tools-BUILT_CONFIG_KEYS) [`COMPOSE_PLUGIN`](#nemo_gym-tasks-harbor-build_tools-COMPOSE_PLUGIN) [`CONTAINERS_CONF_FILE`](#nemo_gym-tasks-harbor-build_tools-CONTAINERS_CONF_FILE) [`CONTAINERS_CONF_OVERRIDE`](#nemo_gym-tasks-harbor-build_tools-CONTAINERS_CONF_OVERRIDE) [`CONTAINERS_CONF_TEXT`](#nemo_gym-tasks-harbor-build_tools-CONTAINERS_CONF_TEXT) [`CONTAINERS_REGISTRIES_CONF`](#nemo_gym-tasks-harbor-build_tools-CONTAINERS_REGISTRIES_CONF) [`HEREDOC_PROBE_DOCKERFILE`](#nemo_gym-tasks-harbor-build_tools-HEREDOC_PROBE_DOCKERFILE) [`HEREDOC_PROBE_TAG`](#nemo_gym-tasks-harbor-build_tools-HEREDOC_PROBE_TAG) [`HEREDOC_REMEDY`](#nemo_gym-tasks-harbor-build_tools-HEREDOC_REMEDY) [`HTTP_403`](#nemo_gym-tasks-harbor-build_tools-HTTP_403) [`IMAGE_FORMAT`](#nemo_gym-tasks-harbor-build_tools-IMAGE_FORMAT) [`LOGGER`](#nemo_gym-tasks-harbor-build_tools-LOGGER) [`REGISTRIES_CONF_FILE`](#nemo_gym-tasks-harbor-build_tools-REGISTRIES_CONF_FILE) [`REGISTRIES_CONF_TEXT`](#nemo_gym-tasks-harbor-build_tools-REGISTRIES_CONF_TEXT) ### API ```python class nemo_gym.tasks.harbor.build_tools.ImageBuilder( tool: str | None = None, repository: str | None = None, push: bool | None = None, platform: str | None = None, storage_root: pathlib.Path | None = None, run: nemo_gym.tasks.harbor.build_plan.Runner | None = None ) ``` Build one context into a named image with docker, buildah or podman, and push it when asked. The image is named as Harbor names it, `hb__<hash>` (see `tag_for`). `repository` is the registry path the name goes under (`registry/path/hb__<hash>`) and the image is pushed there. Without a repository the image stays local under that name in the Docker daemon, the only local store a sandbox provider starts images from, so buildah and podman need a repository. `push` defaults to whether a repository was named. `platform` (`linux/amd64` by default, the platform sandboxes run) is passed to the tool, and an image that comes out for another platform is refused before it is pushed or recorded. `run` is the command seam. docker builds the way Harbor's Docker environment does: `docker compose build` over Harbor's build Compose file and the task's own (`compose_build_argv`), never `docker build`. buildah and podman build the Dockerfile directly (`build_argv`). **`_heredoc_failure`** `str | None = None` --- **`_run`** `= run or _run_default` --- **`_tool_env`** `dict[str, str] | None = None` --- **`platform`** `= platform or DEFAULT_PLATFORM` --- **`push`** `= bool(self.repository) if push is None else push` --- **`repository`** `= repository.rstrip('/') if repository else None` --- **`storage_root`** --- **`tool`** `= tool or detect_tool(self.repository)` --- ```python nemo_gym.tasks.harbor.build_tools.ImageBuilder._check( argv: list[str], what: str, env: dict[str, str] | None = None ) -> subprocess.CompletedProcess ``` ```python nemo_gym.tasks.harbor.build_tools.ImageBuilder._ensure_compose_plugin() -> None ``` Fail before anything is built when `docker compose` (the Compose v2 plugin) is missing. ```python nemo_gym.tasks.harbor.build_tools.ImageBuilder._ensure_heredoc_support( dockerfile: pathlib.Path ) -> None ``` Fail before a Dockerfile with a heredoc is built when this host's buildah or podman cannot parse one. Ubuntu 24.04's packaged buildah 1.33.7 and podman 4.9.3 are linked against openshift/imagebuilder 1.2.5, which reads the body of `RUN <<EOF` as instructions and fails with `Unknown instruction: "ECHO"`. The preflight builds `HEREDOC_PROBE_DOCKERFILE` (`FROM scratch` plus a `COPY` heredoc; no base image is pulled) once per builder with the build's own flags and environment, and a failure is one `BuildError` naming the tool, its version, the Dockerfile and the remedy (`HEREDOC_REMEDY`) instead of one `Unknown instruction` per task. The outcome is cached either way. ```python nemo_gym.tasks.harbor.build_tools.ImageBuilder._storage_flags() -> list[str] ``` ```python nemo_gym.tasks.harbor.build_tools.ImageBuilder._tool_version() -> str ``` `<tool> --version`'s first line, or the tool's name when it will not say. ```python nemo_gym.tasks.harbor.build_tools.ImageBuilder.build( entry: nemo_gym.tasks.harbor.build_plan.BuildPlanEntry ) -> nemo_gym.tasks.harbor.build_plan.BuildResult ``` Build, push when configured, inspect, and return the result to record. Raises `BuildError`. The task's Compose file is read here, not at planning: its `main.build` settings fill the entry (`main_build_settings`) once the file is checked, so a file that does not interpolate, escapes `environment/` or carries a refused key fails this context's build and the caller leaves its tasks out, the way it leaves out a task whose build command fails. ```python nemo_gym.tasks.harbor.build_tools.ImageBuilder.build_argv( entry: nemo_gym.tasks.harbor.build_plan.BuildPlanEntry, tag: str ) -> list[str] ``` The direct Dockerfile build of buildah and podman (docker builds through `compose_build_argv`). `--format docker` builds the image format BuildKit builds, so `SHELL` applies to the `RUN` steps and `HEALTHCHECK` and `ONBUILD` reach the image (the tools' default OCI format has none of the three). `main`'s `build` section of the task's Compose file, which `docker compose build` would merge over Harbor's template, is applied the same way: its context and Dockerfile, `--build-arg` per arg and `--target` (`main_build_settings`). ```python nemo_gym.tasks.harbor.build_tools.ImageBuilder.compose_build_argv( entry: nemo_gym.tasks.harbor.build_plan.BuildPlanEntry, compose_file: pathlib.Path ) -> list[str] ``` Harbor's `docker compose build` line for the context (`_run_docker_compose_command`, [https://github.com/harbor-framework/harbor/blob/v0.22.0/src/harbor/environments/docker/docker.py#L606](https://github.com/harbor-framework/harbor/blob/v0.22.0/src/harbor/environments/docker/docker.py#L606)). The project name stands in for Harbor's session id (Compose's default image name is `<project>-<service>`; `main` is named explicitly, see `harbor_compose_document`); the project directory is the context, so relative paths in the task's Compose file resolve as Harbor resolves them; the files are Harbor's build template, then the task's `docker-compose.yaml` when it has one, in Harbor's order. ```python nemo_gym.tasks.harbor.build_tools.ImageBuilder.inspect( tag: str ) -> dict[str, typing.Any] ``` The image's OCI configuration in the `compose-images.json` record shape (`image` unset). ```python nemo_gym.tasks.harbor.build_tools.ImageBuilder.inspect_argv( tag: str ) -> list[str] ``` ```python nemo_gym.tasks.harbor.build_tools.ImageBuilder.push_argv( tag: str, digest_file: pathlib.Path ) -> list[str] ``` ```python nemo_gym.tasks.harbor.build_tools.ImageBuilder.tag_for( entry: nemo_gym.tasks.harbor.build_plan.BuildPlanEntry ) -> str ``` The image's name, `hb__<hash>`, under the repository when there is one. The hash is the entry's digest, Harbor's environment id of the context, and `hb__<hash>` is the value Harbor exports as `MAIN_IMAGE_NAME` for it (the name a task Compose file that sets `image: ${MAIN_IMAGE_NAME}` builds under; Harbor's own template leaves the tag to Compose). ```python nemo_gym.tasks.harbor.build_tools.ImageBuilder.tool_env() -> dict[str, str] | None ``` The environment buildah and podman commands run with, or `None` (inherit) for docker. The host's `containers.conf` may give every container a default bind mount (this deployment mounts `/etc/localtime` read-only into each one through `containers.conf.d`), and buildah and podman apply those mounts to build steps too, so a Dockerfile that installs `tzdata` (Ubuntu with `python3` pulls it in) fails with `mv: cannot move '/etc/localtime.dpkg-new' to '/etc/localtime': Device or resource busy`. The builder writes a `containers.conf` with no default volumes under its storage root, once, and hands it to the tool as `CONTAINERS_CONF_OVERRIDE`, the layer containers-common applies over the host files. A `CONTAINERS_CONF_OVERRIDE` the caller already set is left as it is. Both tools resolve an unqualified `FROM` (`FROM oven/bun:1`) through the host's `registries.conf`, and Debian's and Ubuntu's stock file defines no unqualified-search registry, so the build fails with `short-name "oven/bun:1" did not resolve to an alias` where `docker compose build` pulls from docker.io. The builder writes a `registries.conf` naming docker.io as the one unqualified-search registry under its storage root and hands it to the tool as `CONTAINERS_REGISTRIES_CONF`. That file replaces the host's main `registries.conf` (the drop-ins under `registries.conf.d`, short-name aliases included, still load), so a site whose main file carries mirrors or blocked registries exports its own `CONTAINERS_REGISTRIES_CONF`, which is left as it is. docker needs neither file and inherits the process environment. ```python nemo_gym.tasks.harbor.build_tools._docker_config( data: dict[str, typing.Any] ) -> dict[str, typing.Any] ``` The set fields of an inspect's `Docker.config` section (the docker-format image configuration), or `{}`. ```python nemo_gym.tasks.harbor.build_tools._proxy_note( output: str, env: dict[str, str] ) -> str ``` Why a registry step may have answered 403, when the tool's output says it did: a proxy the build tool inherits (from its environment, or from its own configuration when the environment names none) can answer 403 to the registry ping unless `no_proxy` carries the registry host. Names only, never values. ```python nemo_gym.tasks.harbor.build_tools._pushed_digest( tool: str, digest_file: pathlib.Path, completed: subprocess.CompletedProcess, run: nemo_gym.tasks.harbor.build_plan.Runner, tag: str ) -> str ``` The manifest digest the registry assigned to the pushed image. ```python nemo_gym.tasks.harbor.build_tools._run_default( argv: list[str], env: dict[str, str] | None = None ) -> subprocess.CompletedProcess ``` ```python nemo_gym.tasks.harbor.build_tools.normalize_inspect( tool: str, data: typing.Any ) -> dict[str, typing.Any] ``` Turn a tool's inspect JSON into the `compose-images.json` record shape (without `image`). buildah reports `{"OCIv1": {"architecture", "os", "config"}, "Docker": {"config"}, "FromImageID"}`; podman and docker report a list with `Architecture`, `Os`, `Config` and `Id`. The image is built in docker format (`IMAGE_FORMAT`), whose `Shell`, `Healthcheck` and `OnBuild` the OCI image configuration cannot hold: buildah reports them under `Docker.config`, read here over the OCI config; docker reports them in `Config`; podman's `Config` is the OCI image configuration (its inspect does not expose `Shell` and `OnBuild`, although the image carries them) and `Healthcheck` sits beside `Config` as a top-level key, read here when `Config` has none. ```python nemo_gym.tasks.harbor.build_tools.proxy_variable_names( env: dict[str, str] ) -> list[str] ``` The proxy-related variables in `env` (`http_proxy`, `HTTPS_PROXY`, `no_proxy`, `all_proxy` and the like), names only: their values can carry credentials. ```python nemo_gym.tasks.harbor.build_tools.BUILT_CONFIG_KEYS = (*RECORDED_CONFIG_KEYS, 'OnBuild', 'Shell') ``` ```python nemo_gym.tasks.harbor.build_tools.COMPOSE_PLUGIN = 'docker-compose-plugin' ``` ```python nemo_gym.tasks.harbor.build_tools.CONTAINERS_CONF_FILE = 'containers.conf' ``` ```python nemo_gym.tasks.harbor.build_tools.CONTAINERS_CONF_OVERRIDE = 'CONTAINERS_CONF_OVERRIDE' ``` ```python nemo_gym.tasks.harbor.build_tools.CONTAINERS_CONF_TEXT = '[containers]\nvolumes = []\n' ``` ```python nemo_gym.tasks.harbor.build_tools.CONTAINERS_REGISTRIES_CONF = 'CONTAINERS_REGISTRIES_CONF' ``` ```python nemo_gym.tasks.harbor.build_tools.HEREDOC_PROBE_DOCKERFILE = 'FROM scratch\nCOPY <