nemo_gym.data.tasks.harbor.build_compose

The credential key a cache_from / cache_to entry carries, if any.

A YAML scalar as Compose passes it to the build: true/false lower-case, else its text.

value (a Compose build path, already interpolated) resolved against base and checked to stay under context with no symlink escape; otherwise BuildError.

One warning per context and variable for a bare ${VAR} the build resolved to an empty string.

services.<service>.build of a Compose document as a mapping (the string form read as its context), or None when the service has none.

The environment of docker compose build, merged as Harbor’s _compose_env_vars merges it (https://github.com/harbor-framework/harbor/blob/v0.22.0/src/harbor/environments/docker/docker.py#L579): the process environment, then the task’s [environment].env (resolve_task_env), then Harbor’s infrastructure variables, which win: MAIN_IMAGE_NAME, CONTEXT_DIR, CPUS and MEMORY (as <n>M) from the task’s resources, and the legacy HOST_*_PATH / ENV_*_PATH log mounts, plus the two the loader defines for the same file (nemo_gym.data.tasks.harbor.image_configs.compose_variables): TEST_DIR and COMPOSE_PROJECT_NAME (hb__<hash>, the image name; a stable stand-in, not Harbor’s per-trial project name), so the build and the seed interpolate a task’s file from the same set. A task variable named like an infrastructure one is ignored with a warning, as merge_compose_env does. DOCKER_DEFAULT_PLATFORM carries builds.platform, the one setting Harbor has no equivalent of.

The variables docker compose interpolates the task’s Compose file with for a command run with env: that environment over the project directory’s .env file (Compose reads it from --project-directory, which Harbor sets to context; a variable set in the environment wins over the file). .env values are read as Compose’s dotenv reader reads them (compose-go/dotenv): an unquoted or double-quoted value is itself interpolated, in file order, against the environment and then the lines above it, with the document’s ${VAR}, ${VAR:-default} and $VAR forms and $$ (or \$ in double quotes) for a literal dollar; a single-quoted value is literal. A reference is resolved as it is in the document (substitute_with_empty, the loader’s rule): NET=${X:-host} in .env is host to the check as it is to Compose, a bare ${X} nobody sets is an empty string with one warning naming tasks, and ${X:?msg} is refused. COMPOSE_ENV_FILES set in env is refused: it points Compose at other files than the .env read here.

The Compose file Harbor’s Docker environment builds a task with, resolved for one build context.

Harbor’s docker-compose-build.yaml (v0.22.0, quoted in full; it is the first -f of every docker compose command Harbor runs for a task, see _docker_compose_paths at https://github.com/harbor-framework/harbor/blob/v0.22.0/src/harbor/environments/docker/docker.py#L316):

services:
  main:
    build:
      context: ${CONTEXT_DIR}
    pull_policy: build
    command: [ "sh", "-c", "sleep infinity" ]

This is that file with CONTEXT_DIR resolved to the task’s environment/ folder and one addition: image set to hb__<hash>, the value Harbor exports as MAIN_IMAGE_NAME (ComposeInfraEnvVars.main_image_name at https://github.com/harbor-framework/harbor/blob/v0.22.0/src/harbor/environments/docker/docker.py#L238). Harbor’s template sets no image: its compose build tags <session>-main unless the task’s own Compose file says image: ${MAIN_IMAGE_NAME}. Gym needs a stable name to inspect, push and record the result under, so it sets that value itself. No dockerfile is named, as in Harbor’s file: Compose defaults to Dockerfile, and naming it would clash with a task file’s dockerfile_inline (Compose refuses a service declaring both). Every other field is Harbor’s, verbatim.

hb__<digest> sanitised as Harbor sanitises it, under repository when there is one: the name ImageBuilder.tag_for builds under and MAIN_IMAGE_NAME in the build’s environment.

Gym’s images override for docker compose build: image per built service, main included.

Harbor passes its template, then the task’s file, then its own generated overrides (docker.py#L363-L371 at v0.22.0). Gym’s one override, passed last, names every service it builds: main as hb__<hash> (the template already says so) and each sidecar as hb__<hash>-<service>. Last wins in Compose, so a sidecar’s own image: in the task file (kumo’s ${MAIN_IMAGE_NAME}_verifier) is overridden without a docker tag step, and a sidecar without one gets Gym’s name instead of Compose’s <project>-<service> default, which under the project name Gym passes is the same string.

A task’s [environment].env the way Harbor hands it to docker compose (resolve_env_vars, https://github.com/harbor-framework/harbor/blob/v0.22.0/src/harbor/utils/env.py#L94): a whole value of the form ${VAR} or ${VAR:-default} is read from the process environment, the default standing in when the variable is unset; any other value is literal. A ${VAR} with no default and no value fails the build naming it, as Harbor fails the trial.

The entry’s service’s build section of the task’s Compose file as BuildPlanEntry fields, or {} without one.

document is the file as validate_task_compose returned it for env, the build’s environment (compose_build_env): interpolated as Compose interpolates it, its paths checked to stay under environment/. This is what docker compose build applies from the task file (over Harbor’s template for main), read here so the direct build of buildah and podman applies the same: the build context (a path, or the string form of the section) and dockerfile, resolved under environment/ (_contained_build_path again, so a caller handing in an unchecked document still cannot escape it); args as a mapping or Compose’s KEY=value list, where a mapping value of null and a bare list KEY alike take the variable’s value from the build’s environment and are dropped when it is unset, as Compose drops them, so the Dockerfile’s own ARG default stands; target; and whether a dockerfile_inline is set. args of another shape is a BuildError.

The task’s own Compose file in context: docker-compose.yaml, the one name Harbor reads, or None.

The loader accepts the other spellings (docker-compose.yml, compose.yaml, compose.yml) at run time; Harbor’s Docker environment never appends them to a build, so a context shipping one is refused here rather than built from a file Harbor would ignore, under the same hb__<hash> as Harbor’s build.

Check the task’s Compose file before docker compose build runs it on the build host, and return it interpolated (None when the context ships none). Anything refused raises BuildError naming the service and key.

The file is third-party content and the build host often holds the push credentials, so what it may ask of the host is bounded to what a legitimate task build needs: a service with a build section may carry args, a target, a context and a Dockerfile under environment/ (symlinks resolved, like the loader’s env_file rule), or a dockerfile_inline. Refused on any service with a build section: secrets, ssh, extra_hosts, entitlements and privileged (BuildKit’s --allow network.host / security.insecure, which let a RUN --network=host line reach the build host’s network), network: host, additional_contexts other than service: references, and cache_from / cache_to entries carrying credentials; refused anywhere: a top-level include and an extends with a file, which pull further files in unchecked. main’s image has to resolve to MAIN_IMAGE_NAME when set and main is among the services being built (builds_main): ${MAIN_IMAGE_NAME}, ${MAIN_IMAGE_NAME:-...} and ${MAIN_IMAGE_NAME-...} all do, since Harbor always exports the variable; otherwise the build would be tagged under another name than the one recorded. A sidecar-only build (a prebuilt, pull-mode or overlay main, or one already recorded) neither reads nor tags main’s image, so its spelling is not checked, as Harbor’s prebuilt mode never reads it.

Compose interpolates the file against the command’s environment before it reads a single key, so the checks run on the document interpolated the same way: env is the environment the build runs with (compose_build_env), the project directory’s .env under it with its values expanded as Compose’s dotenv reader expands them (compose_interpolation_variables), with Compose’s ${VAR}, ${VAR:-default}, ${VAR-default}, ${VAR:?error}, ${VAR:+alt} and $VAR forms on every string. Otherwise network: ${X:-host} or cache_to: [type=gha,${X:-token}=abc] would pass a check on the raw text and resolve to the refused value at build time. A bare ${VAR} nobody sets is an empty string with one warning naming tasks and the variable, as docker compose build makes it and as the loader resolves the same file at seed (substitute_with_empty), so the document checked is the document built; ${VAR:?message} is refused with the message. main’s image is interpolated the same way before it is compared, so any spelling Compose resolves to the main image passes and a literal other image, or a reference to another variable, is refused with what it resolves to.

A sidecar with a build section passes the same checks as main’s and is built (plan_builds), as Harbor builds it. Its image:, when it has one, is accepted whatever it says: the build is tagged with Gym’s name for it (ImageBuilder.tag_for) and the seed document is filled from the record by service name, so no rule on its spelling is needed.

Write Harbor’s build template (harbor_compose_document) and Gym’s images override (images_compose_document) under folder and return their paths, in the order they are passed.