nemo_gym.data.tasks.harbor.build_records

value (a parsed YAML section) as one line of JSON that is the same for the same section: keys sorted and read as text (YAML allows other scalars as keys), and a scalar JSON has no form for, a date say, as its text.

(service, inputs digest), sorted by service, for each service whose image entry’s build reads: the service:<name> references of its build.additional_contexts (a mapping, or Compose’s name=service:x list) that name a service with a build section of document. Each is settled as the entry is and digested with its own dependencies (build_inputs_digest), so the digest is transitive: a change to db’s inputs changes main’s when main builds FROM base with base: service:db, and docker compose build would build a different main. A service already on the path (seen: Compose refuses the cycle at build) adds nothing. Only docker compose build honours such references (the direct tools refuse them, ImageBuilder._refuse_on_direct_path), and only it follows them to a rebuild of the consumer.

The inputs digest of a settled entry (settle_entry): inputs_digest of its context (every entry of the tree with its mode, content or link target, nothing skipped) plus what the build reads outside the tree: the service’s build section of the task’s Compose file as a whole, as it stands after interpolation (Compose interpolates every string of the file against the build’s environment, so an inline Dockerfile, a label or a cache setting changes the image while no file does), its two paths by their resolved forms relative to the context (the text may spell them through CONTEXT_DIR, an absolute path that differs between hosts holding the same folder), the build arguments as resolved (a null or bare argument is read from the environment, which the section does not show) and the inputs digest of each service whose image the build reads through a service: reference of its additional_contexts (BuildPlanEntry.dependency_digests: the section names the service, not what that service is built from). The services of one context share the tree and differ in the section, so each image carries its own. Recorded in the built image’s configuration record and compared before a recorded image is reused (build_inputs_mismatch).

Why the image the built-images.json entry record holds for entry’s service was built from other inputs than the context holds now, or None when the inputs match and the recorded image may be reused.

The context’s Harbor hash names the image and keys the record, but it covers file names and bytes only; the inputs_digest in the image’s compose-images.json record (image_configs, written by the build with build_inputs_digest) covers the rest, so a context whose name is unchanged is still rebuilt under that name when its inputs are not. An image whose record has no digest (written before it was recorded, or missing) counts as changed. Settling the entry can fail as the build would (BuildError: a Compose file that does not interpolate or is refused); that is reported as the reason too, so the caller rebuilds, and the rebuild reports the error in full.

os/arch of a platform string, with a variant such as linux/arm64/v8 dropped.

Record one built image next to the task folders: under its context digest and service in built-images.json and by image in compose-images.json. Existing entries are kept unless overwrite.

The digest’s entry is merged, not replaced: services[<service>] is set to this build’s image, tool and pushed (the other services keep theirs: one context’s services may be built by differently configured runs, and a pushed main must not vouch for a local sidecar or the reverse), tasks unioned, and the top-level keys older readers use kept equal to main’s: image (absent while main is not recorded) with tool and pushed (the first recorded service’s while main is not recorded). A service recorded once keeps its image unless overwrite, like compose-images.json keeps its entries.

Both files are merged and replaced under their sidecar locks (record_lock), the built-images lock first and the compose-images lock inside it, always in that order; record_image_configs takes only the compose-images lock, so the two writers cannot deadlock and neither loses the other’s entries. The configuration lands before the build entry, as before, so a reader that finds the entry finds its configuration.

Add tasks to the forced_tasks of the context digest’s entry in built-images.json: the tasks whose declared docker_image a prepare under builds.force built over, when the context’s main was already recorded and is reused (a fresh build records them through record_built_image). No entry, no write.

Why the image a built-images.json entry records for service cannot serve this builder, or None when it can.

A service the entry does not record is missing. A local (unpushed) image serves only local builds; a pushed one serves both, judged by that service’s own pushed flag (recorded_service_records: the entry’s flag stands in for a record written before the flags were kept per service), since the services of one context may have been built by differently configured runs. The recorded platform has to be the one asked for.

entry with its service’s build section of the task’s Compose file applied, as ImageBuilder.build_many applies it before the build command: the file is checked and interpolated against the build’s own environment (compose_build_env, for the image name and platform a builder with repository and platform uses), and its settings fill the entry (settle_with_document). Raises BuildError where the build would.

entry with its service’s build section of the checked and interpolated Compose document applied (service_build_settings) and the inputs digests of the services it builds from filled in (BuildPlanEntry.dependency_digests, see _dependency_digests). What ImageBuilder.build_many settles the entries it builds with, and settle_entry the one it checks, so the digest recorded and the digest compared are computed alike.