Skip to navigation

nemo_gym.data.tasks.harbor.prepare_builds

View as Markdown

The build step of prepare and the two prepare-time Compose field checks.

Before any build runs, each build context’s raw Compose file is checked against the sandbox adapter’s field allow-list, and after the tasks load each task’s resolved document is checked the same way, so a task the adapter would refuse at seed is refused at prepare with the same message. The build step plans the builds, drops refused contexts, reuses recorded images that fit, honours the builds table, constructs the ImageBuilder and collects per-task failure reasons. prepare_target in nemo_gym.data.tasks.harbor.prepare calls it.

Module Contents

Functions

NameDescription
_build_dockerfilesBuild and record the images of the selected tasks whose Dockerfile needs a build.
refused_before_buildThe key refusals of the tasks plan would build, read from their Compose files before any build runs.

API

nemo_gym.data.tasks.harbor.prepare_builds._build_dockerfiles(
folder: pathlib.Path,
select
) -> dict[str, str]

Build and record the images of the selected tasks whose Dockerfile needs a build.

With builds disabled, the tasks that need one are left for the loader to skip with its message, a task whose separate verifier’s tests/Dockerfile is unbuilt included (the loader refuses an unrecorded verifier image the way it refuses an unrecorded main), and a recorded image that does not fit the run (another platform, or local when the run pushes) leaves its tasks out with the reason, as the enabled path would rebuild it. When every context has a usable record nothing is built and no build tool is looked for. Otherwise a missing builder stops the preparation, naming the tasks. A failed build does not: each context (every service it builds) is built on its own, and the tasks of one that fails are returned by id with the reason, for the caller to leave out the way it leaves out a task that does not load. A task in separate verifier mode may own two contexts, environment/ and tests/ (its verifier’s image); a failure of either leaves the task out. A planned task whose Compose file the sandbox adapter would refuse whatever the build gave (refused_before_build) is returned the same way, and its contexts are neither built nor pushed for it; a context it shares with an eligible task (a separate verifier’s tests/, byte-identical across tasks whose environment/ differ) is still built for that task. A task whose folder has another task’s Harbor hash but other build inputs (plan_builds’ conflicts) is left out with the reason while that other task stays in: when the other task is itself refused in the same round, the plan is made again without it, and this task’s folder is the one built under the name if it is first then.

nemo_gym.data.tasks.harbor.prepare_builds.refused_before_build(

The key refusals of the tasks plan would build, read from their Compose files before any build runs.

The authoritative check is check_task_compose_fields (record_compose_images runs it on the loader’s document once the tasks load); this screen is its first step, the key allow-list (unsupported_compose_fields), which needs only key names, which interpolation and env_file inlining do not add. It reads the raw file the way the loader’s document will look: pull_policy dropped, the build: of every service the plan builds from the context dropped (they are about to be built and will run from their recorded images) and env_file set aside, as the loader folds it into environment. A task it names is neither built nor pushed for an image no episode would start, and a build that would fail for it cannot stop the preparation, with the reason spelled as the loader-time check spells it. A file that does not parse is left to the loader to report in its own words. The separate verifier’s tests/ context is not checked: its image starts as a single sandbox, never through the adapter. Its entry is the one with BuildPlanEntry.verifier set; an ordinary sidecar that happens to be named verifier (kumo’s) is a service of the environment/ document like any other, built and substituted here.

When the plan does not build main (a pull or overlay Dockerfile beside a built sidecar), a main build: that only restates Harbor’s own build of that Dockerfile is dropped as the loader drops it (restated_main_build_reason, the rule of without_restated_main_build, with ${CONTEXT_DIR} accepted as the raw file spells the context), and a key Gym cannot apply without a build is the task’s reason here too, spelled as the loader spells it.