nemo_gym.data.tasks.harbor.compose_overlay

Bases: ValueError

An overlay file, its merge or the prompt template cannot be applied as written.

Bases: BaseModel

The compose_overlay block of the Harbor environment server’s configuration.

files are Compose files merged, in order, onto every task’s Compose document after the task’s own file (Harbor’s extra_docker_compose); mcp_servers are MCP entries merged by name with the task’s [[environment.mcp_servers]], the overlay’s winning on a name (Harbor’s agents[].mcp_servers); prompt_template is a Jinja template with an {{ instruction }} variable, rendered around the instruction of the episode’s first turn (Harbor’s prompt_template_path). Every path must exist when the block is read: Harbor refuses a missing extra Compose file the same way (FileNotFoundError in _normalize_extra_docker_compose_paths), and a template that does not name instruction or does not parse is refused before any episode, as Harbor’s render_prompt_template refuses it. The template is read and compiled once, when the block is read (compile_prompt_template); every episode renders the compiled template (render_instruction), so the file is not re-read per episode and an edit to it under a running server is not seen until the server is started again.

nemo_gym.data.tasks.harbor.compose_overlay.HarborComposeOverlay.compile_template() -> 'HarborComposeOverlay'

nemo_gym.data.tasks.harbor.compose_overlay.HarborComposeOverlay.existing_files(
    files: list[pathlib.Path]
) -> list[pathlib.Path]

nemo_gym.data.tasks.harbor.compose_overlay.HarborComposeOverlay.readable_template(
    path: pathlib.Path | None
) -> pathlib.Path | None

nemo_gym.data.tasks.harbor.compose_overlay.HarborComposeOverlay.render_instruction(
    instruction: str
) -> str

Bases: dict

A mapping tagged !override: it replaces the earlier files’ value instead of merging into it.

Bases: str

A scalar tagged !override. A scalar replaces the earlier value whether tagged or not, so the tag changes nothing where the value stays a scalar; it is kept for the fields whose scalar form is a one-entry sequence (env_file: !override /path/x.env, tmpfs, dns), which the normalization before the merge turns into an OverrideSequence, so the one entry replaces the earlier files’ lists instead of appending to them, as the tagged list form does. _plain leaves a plain str in the merged document.

Bases: list

A sequence tagged !override: it replaces the earlier files’ value instead of appending to it.

Bases: SafeLoader

yaml.SafeLoader that reads Compose’s !reset and !override tags.

A !reset value: the attribute the earlier files set is removed; the value written after the tag is not read (Compose ignores it too).

nemo_gym.data.tasks.harbor.compose_overlay._Reset.__repr__() -> str

The jinja2 module, imported when a prompt template is read: the Harbor environment server’s own requirements install it (Harbor renders its templates with Jinja2 too); the loader and the harnesses that import this module for the merge and the MCP rule do not need it.

A keyed field (environment, labels, extra_hosts, …) as a mapping, whichever form it was written in: a list of KEY=value (KEY alone is None; host:ip for extra_hosts).

A list-form environment entry, a bare NAME that variables sets resolved to NAME=value.

value without markers: !override wrappers become plain mappings, sequences and strings, and a !reset under a key no earlier file set leaves the key absent, as Compose leaves it.

A field of the wrong shape, named by its path and the value’s type, never the value: a mistyped environment: API_KEY=... would otherwise print the key in prepare’s skipped.json and the seed’s reason.

items with one entry per key, the last one written, in the order its key first appeared.

What went wrong and where, without PyYAML’s snippet of the offending line: an overlay file commonly carries the deployment’s keys, and the message reaches prepare’s skipped.json and the seed’s failure reason. A MarkedYAMLError gives its context and problem (PyYAML’s own phrases, no source text) with the line and column; any other error only its kind.

Each overlay service’s env_file entries replaced by the absolute paths they resolve to against project_dir (overlay_env_file_resolver, resolve_env_file_entries), the files neither read nor checked to exist yet: Compose resolves each file’s paths before the merge and reads the env files of the merged document once after it, the merged environment winning, so a file a later overlay file drops (env_file: !reset []) is never required to exist and a missing one that survives the merge is reported at that reading. The markers are kept for the merge: env_file: !reset [] and a service tagged !reset pass through, an !override list keeps its tag and an !override scalar becomes an !override list of its one path (so it replaces the earlier files’ lists, as Compose applies the tag), and environment is not touched (its own !reset and !override markers, and its normalization, are the merge’s and the final reading’s). Bind sources are not rewritten: the volume planner resolves a relative source of any file against the same project directory, which is where Compose puts it.

The template at path compiled once it passes validate_prompt_template: a jinja2.Template from an environment with StrictUndefined, as Harbor’s render_prompt_template builds one per render. The block compiles its template when it is read and renders the compiled one per episode, so the file is read and parsed once, not twice per episode on the event loop.

document with substitute applied to every string, the !reset and !override markers kept.

Compose interpolates each -f file against the command’s environment before merging, so the overlay files take the same substitute as the task’s file (the task’s resolver, with the deployment’s values when the server has them).

Read one overlay Compose file (load_overlay_text); a file that cannot be read raises ComposeOverlayError naming it.

Parse one overlay Compose document with its !reset and !override tags, as a mapping.

Top-level extension fields (x-*, the anchors a file keeps its shared blocks under) are dropped, as Compose ignores them; their aliases were expanded at parse time. Anything that is not a mapping, or does not parse, raises ComposeOverlayError naming source.

overlay merged onto base as Compose merges a later -f file onto the earlier ones.

Mappings merge key by key; sequences are appended, the keyed ones (ports by host, published port, target and protocol; volumes by target; expose, cap_add, env_file, … by value) unique on their key with the later entry winning; environment, labels, extra_hosts, sysctls and build.args merge as mappings in either written form; depends_on and a service’s networks merge as mappings (a list becomes {name: {condition: service_started}} and {name: null}); command, entrypoint and healthcheck.test are replaced whole; any other scalar, or a value of another shape, is the later file’s. A key written with no value in a later file (a bare memory_net: under networks re-declaring one an earlier file configured) keeps the earlier value, as compose-go keeps it for a nil override, except on the replaced paths. A !reset removes the key the earlier files set; an !override mapping or sequence replaces instead of merging. Neither input is modified; the result carries no markers.

The task’s MCP entries with the overlay’s merged in by name, in Harbor’s order and with Harbor’s rule.

trial.py _init_agent (v0.22.0) builds {server.name: server} over the task’s environment.mcp_servers followed by the job’s agent.mcp_servers and hands the agent the values: a job entry replaces a task entry of the same name, in the task entry’s position; a new name goes last.

The resolver of an overlay file’s env_file entries: a relative path against project_dir, the Compose project directory (the task’s environment folder, Harbor’s --project-directory; Compose resolves every -f file’s relative paths against it, not against the file that wrote them), an absolute path as written, any readable file accepted. The overlay files are the operator’s own, named in the run config, so the containment rule of a task’s entries (third-party content that prepare digested, under_task_folder) does not apply, as it does not under Harbor, where Compose reads an extra file’s env files from wherever they are.

The overlay’s Compose files, [] without an overlay.

instruction rendered into template (compile_prompt_template), path named in the errors: instruction is the one variable, so a variable the template names and nothing defines raises ComposeOverlayError (StrictUndefined), as under Harbor.

instruction rendered into the template at path, as Harbor renders it (render_prompt_template in harbor/utils/templating.py at v0.22.0): Jinja2 with StrictUndefined, instruction the one variable. Reads and compiles the file on every call; the server renders a block’s compiled template instead (HarborComposeOverlay.render_instruction).

document with each overlay service’s pass-through environment entries resolved against variables: a bare NAME in the list form becomes NAME=value and a NAME: with no value in the mapping form becomes NAME: value for every name variables sets; a name it does not set stays bare, and the markers are kept (a !reset entry, an !override list or mapping, a service tagged either).

Compose resolves the pass-through spelling against the command’s environment, the one every -f file is interpolated with, so under Harbor it follows the order of ${NAME} in every file. Gym reads the merged document’s pass-through entries once, after the merge, against the task’s variables (inline_env_files), which is the task’s own file’s order; an overlay file’s entries follow the overlay’s order (the deployment’s values above the task’s sources) only when they are resolved before the merge, here, with the variables interpolate_overlay took for the same file. A name nobody sets stays bare and is read after the merge as the task’s entries are: unset in the container.

The Compose document of a task that declares no Compose file: Harbor’s own main service, nothing else.

Harbor starts every task through Compose (its base file plus the build or prebuilt file define main), so an extra Compose file extends a plain image task the same way it extends a Compose task. The sandbox adapter fills main’s image and command from the task, as it does for a task file that leaves them out.

For a Dockerfile task whose image configuration the loader resolved, preserve its ENTRYPOINT (including an explicit clear) and final WORKDIR instead of inheriting the base image’s settings. The server’s setup hook runs any Dockerfile RUN lines and installs the image’s startup user before the command and healthcheck. The adapter defers the working directory until that setup finishes, so it cannot create a root-owned directory ahead of a non-root Dockerfile WORKDIR step. No user is written here: numeric users need resolution inside the running image, while an explicit user in a job overlay still wins over the installed default.

[agent].user and [environment].workdir apply to the agent’s later commands, independently of startup. A prebuilt docker_image task without a Dockerfile is left to image resolution; without a task, main is empty.

The template’s text, once it parses and names instruction; otherwise ComposeOverlayError.

Harbor’s render_prompt_template refuses a template that does not use the instruction variable (the task’s instruction would be lost) and renders with StrictUndefined, so any other variable fails.

A copy of responses_create_params with render applied to the instruction: the first user message’s text (a string content, or the text of its first input_text part). The row’s instruction is that message, as the loader materializes it; a request without one raises ComposeOverlayError.