Multi-Container Tasks

Some tasks need more than one container: a database the agent queries, an API it calls, a simulated user, or a service that records what the agent did. Describe them with a Docker Compose file in the task’s environment/ folder. The harbor Environment Server starts each service as its own sandbox, joins them into one group for the episode, and stops the group when the episode ends.

Use Compose when the task needs a process with its own image, or one the agent must not control. A tool the agent calls can often run in the agent’s sandbox instead (see A task with its own MCP tool).

Layout

my-task/
├── task.toml
├── instruction.md
├── environment/
│   ├── Dockerfile
│   ├── docker-compose.yaml
│   └── seed.json
├── tests/test.sh
└── solution/solve.sh

Name the file environment/docker-compose.yaml, the name Harbor reads. Service main is the agent’s sandbox, and every other service is a sidecar:

services:
  main:
    image: ${MAIN_IMAGE_NAME}
    depends_on:
      - api
  api:
    image: ghcr.io/example/aw-api:1.0
    volumes:
      - ./seed.json:/app/seed.json:ro
    expose:
      - "8080"

${MAIN_IMAGE_NAME} is the task’s own image. Services reach each other by name, so the agent calls http://api:8080. A sidecar with a build: section is built once at prepare, like a build-mode Dockerfile (see Dockerfile modes and Harbor image builds).

Prepare (gym eval run or gym dataset validate) resolves the file, checks every service, and records each image’s configuration in compose-images.json at the dataset root. A task it cannot run is left out and listed in skipped.json with the reason, and gym eval run stops unless you pass --skip-unloadable. The server checks the document again at seed.

artifacts entries and [[verifier.collect]] hooks can name a service. A sidecar that writes a file for test.sh needs Sidecar inputs.

Run on OpenSandbox

The shipped OpenSandbox config does not enable networking between sandboxes. Add a deployment file and pass it with --config:

sandbox:
  opensandbox:
    networking:
      enabled: true
    runtime_requirements:
      shm_size_metadata_key: <cluster label for shm size>
    shared_storage:
      host_path: /mnt/shared
  volume_sources:
    pgdata: datasets/pgdata

gym dataset validate my-dataset --sandbox opensandbox --config compose-deployment.yaml

See OpenSandbox and Compose Services.

Template variables

Prepare resolves ${NAME} the way Harbor’s docker compose does, and the server resolves it again at seed. The first source that defines a name wins:

1. Gym’s infrastructure variables, below.
2. The task’s [environment].env. A whole value ${VAR} or ${VAR:-default} reads the process environment. An unset VAR with no default leaves the task out.
3. The process environment of the command that runs prepare or the server.
4. environment/.env, read as Compose reads it.
5. At seed only, the server’s compose_variables.
6. The default in the expression, such as ${VAR:-default}.

At prepare, a name no source sets becomes an empty string, with a warning. At seed it fails the episode before any sandbox starts.

Values that belong to the deployment, such as a sidecar’s model endpoint, go in the server’s compose_variables. Each entry takes one source: a literal value, a server process variable env, or a model_server, whose sandbox-reachable URL becomes the value:

harbor_environment_server:
  environment_servers:
    harbor:
      compose_variables:
        SIM_USER_BASE_URL: {model_server: user_model}
        SIM_USER_API_KEY: {env: USER_MODEL_API_KEY}
        SIM_USER_MODEL: {value: gpt-5.5}

A service’s env_file entries are read at prepare and inlined, with no ${VAR} interpolation, into its environment, which wins over them. The file must sit under the task’s environment/ folder and must not be a symbolic link. A missing file leaves the task out unless the entry says required: false.

Service fields

Beyond image, command, entrypoint, environment, working_dir, user, depends_on, healthcheck, expose, labels, cap_add, shm_size and network_mode: service:<name>, these fields have a defined meaning. Any other service key, such as privileged or secrets, leaves the task out. So does a top-level key other than name, services, networks and volumes, such as version or an x- block. A depends_on entry with required: false or restart: true, or a user with a group, such as 1000:1000, fails when the group starts.

[environment.healthcheck] runs at seed as main’s user: when one is set, as a decimal string’s integer user ID; with no user: it runs as the image’s user. The agent keeps its own user.

Volumes

Anonymous volumes, tmpfs volumes, bind propagation and volume drivers leave the task out. Keep data that test.sh must trust in tests/, which the server uploads after the agent’s turn.

Job-level Compose overlay

The compose_overlay setting of the harbor Environment Server attaches services, tools and a prompt template to every task of a run without editing the tasks, as Harbor’s extra_docker_compose, agents[].mcp_servers and prompt_template_path job settings do. Use it to run one backend, such as an agent memory service, beside every task. It goes in the server block (see Configure the server):

harbor_environment_server:
  environment_servers:
    harbor:
      compose_overlay:
        files: [/abs/path/memory-compose.yaml, /abs/path/memory-glue.yaml]
        mcp_servers:
          - {name: memory, transport: streamable-http, url: "http://memory:8888/mcp/"}
        prompt_template: /abs/path/memory_prompt.md.j2

• Write absolute paths. Relative paths inside an overlay file resolve against the task’s environment/ folder.
• In an overlay file, compose_variables win over the task’s own sources, so a task cannot redirect an overlay sidecar.
• A task service with an overlay service’s name merges with it, with a warning. Tag the overlay service !override to use its definition alone.
• Prepare reads the block too, records the overlay’s images and checks each merged document. Put it in env.yaml or a --config file that both prepare and the server read, and prepare again after you change it. Image references must resolve at prepare, without compose_variables.
• For a Dockerfile task without its own Compose file, attaching services keeps the image’s startup user and working directory: the entrypoint and image health check run as the Dockerfile’s final USER and WORKDIR, after its RUN lines, which can already write to named and external volumes. [agent].user and [environment].workdir apply to the agent’s commands. An explicit services.main.user or services.main.working_dir in an overlay file overrides the startup default.
• Restart the servers after you edit the template. Prepare also compiles it, so install jinja2>=3.1 where you run gym (the ray extra includes it).