nemo_gym.sandbox.compose_config
nemo_gym.sandbox.compose_config
Turn a published Harbor Compose overlay into a document the sandbox Compose adapter can start.
Harbor publishes environment/docker-compose.yaml as an overlay on its own base file: the
main service is the task image, sidecars are prebuilt images. Starting it without Docker
needs each image’s recorded OCI configuration (entrypoint, command, user, exposed ports,
healthcheck), which image_configs supplies, keyed by image reference.
Adapted from Gym’s Terminal Bench 4 server (resources_servers/terminal_bench_4/compose_config.py).
Module Contents
Functions
Data
API
The refusal’s reason: the block service_name runs on has no shared storage, and where that is set.
Whether value is exactly one URL whose host is a Compose service, the shape resolve_environment rewrites.
A compound value (Kafka’s comma-separated KAFKA_ADVERTISED_LISTENERS) parses to a hostname too, but
its port text is not a number and the adapter would fail reading it; it is left as written.
Keep a sidecar on its image’s default non-root user without host-file injection.
A non-root process cannot edit /etc/hosts, so peer names are resolved into the
environment variables that carry http://<service> URLs instead, and the explicit
user copied from image metadata is dropped so the provider does not try to su
to the user it already runs as. Such a sidecar cannot take extra_hosts
(check_sidecar_extra_hosts raises first). Mutates and returns document.
Refuse extra_hosts on a sidecar the group runs without host injection, naming the service.
A non-root sidecar gets no hosts file entries (see adapt_non_root_sidecars), so host entries it
declares would be silently lost; the adapter rejects that document at seed. Prepare runs this check once
the image configurations are recorded, so the task fails there, with the same reason.
The platform a deployment runs, which a service’s platform must match: the sandbox provider options’
platform (OpenSandbox’s {os, arch}) when set, else linux/amd64, the one prepare records images for.
Prepare and the server read it from the same sandbox_provider_options so the two checks agree: the
os/arch form of sandbox_platform, the reader the image checks use.
The entries of volume_sources a planned Compose document resolves through it, and nothing else.
The adapter treats every volume name it finds in its mapping as externally managed (one shared path,
no per-group directory, no copy-up), so a deployment-wide mapping handed over whole would make an
ordinary volumes: {pgdata: null} of one task share the directory another task maps as external. Only
the names this document declares external: true and mounts, and the host paths of its remaining
(foreign) bind mounts, are kept; a key the document never mounts, or mounts as an ordinary named volume,
is dropped, and that volume stays private to the group, as Harbor’s per-trial Compose project keeps it.
The sidecars whose recorded image USER is not root, with that user.
These are the services adapt_non_root_sidecars keeps on their own user without host injection;
prepare uses the same predicate so what it accepts is what the group runs. image_configs is read
by reference and by pinned image, so the published document and the resolved one both work.
OpenSandbox takes a service’s shared-memory size as a label. Mutates and returns document.
Decide how each service’s volumes run in a sandbox group. Mutates and returns per-service uploads.
A bind mount whose source is a file or folder of the task (under root, default base_dir;
relative paths resolve against base_dir, as Compose resolves them against the Compose file)
becomes an upload into the service once it starts: the entry leaves volumes and joins
x-sandbox.uploads as {"source": <host path>, "target": <container path>}. With a task root
given, such a source must be one the task digest covers (is_covered:
under environment/, tests/, solution/ or steps/, or one of the top-level task files, and
not a name the digest skips), or it raises naming the path: an upload the digest does not see could change
without changing the task’s identity. Named volumes declared at top level stay, in long syntax, for the provider’s shared
storage; a volume declared external: true, and a bind source outside the task, resolve through
volume_sources (the sandbox block’s mapping of volume name or host path to a shared-storage path)
and raise ComposeVolumeError naming the volume when it is missing. A sidecar that reserves GPUs
through deploy.resources resolves through gpu_volume_sources instead when that is given (the
gpu_sandbox_provider block’s mapping, the block the server runs it on; main follows the task, as
check_compose_fields routes the capability check). A bind of Harbor’s /logs tree onto main
leaves volumes and joins x-sandbox.logs as {"source": <logs path>, "target": <container path>}:
the adapter creates the source in the agent’s sandbox before the service’s command, hooks and healthcheck,
and links a target elsewhere to it, as Docker creates both of Harbor’s bind targets at container create.
A sidecar that binds a path of the tree shares it with main the way Harbor’s one host folder is shared:
the path becomes a named volume on the provider’s shared storage (declared as nemo-gym-<path>, with
nocopy; a second path that spells the same name, /logs/a/b beside /logs/a-b, gets a digest of its
spelling appended), mounted at the sidecar’s target and at the path itself in main, so what the sidecar
writes there is what the verifier collects. A logs path with a .. segment (/logs/../etc) raises naming the
path, on main or a sidecar: it is used as written. shared_storage says whether the deployment’s provider has shared
storage (gpu_shared_storage for the services gpu_volume_sources routes): False refuses every
named volume, mapped source and shared logs bind at prepare, naming the setting, instead of at every seed;
None leaves that to the adapter. A source the task does not contain, or a symlink that leaves it, raises naming the
path; so does a bound folder that contains a link, and a source reached through a link with an absolute
target (see absolute_link). A service that already carries x-sandbox raises naming the field:
that key is this planner’s output, not task input. A null top-level declaration (volumes: {pgdata: null}) is normalised to {}, the form the adapter reads.
Apply Harbor’s prebuilt-image base and normalize the published task overlay.
platform is the (os, architecture) the sandboxes run (sandbox_platform); a service whose
recorded image is for another platform is refused, naming both.
The (os, architecture) the sandboxes run: the OpenSandbox platform provider option
({os, arch}, see nemo_gym.sandbox.providers.opensandbox) when a deployment sets one, otherwise
linux/amd64, the default of every provider.