nemo_gym.sandbox.compose_fields

What a Compose service may say, and the shape each field takes before the adapter reads it.

The field allow-list of the sandbox Compose adapter and its message per offending key, the normalisation of deploy resources, devices and gpus, restart, pull_policy, platform, extra_hosts, hooks, networks and the provider service options, and check_compose_fields, which runs them all over a document and is a fixed point of its own output. The two Compose scalar and environment shape helpers, the memory size parser and the interpolation error class live here because the field checks, the resolver and the interpolation code in nemo_gym.data.tasks.harbor.compose_variables all read them, and this module sits below both. Nothing here reads a task or a record.

Module Contents

Classes

Name	Description
`ComposeFieldError`	A Compose service field prepare cannot run as written; `field` is the Compose key it names.
`ComposeSubstitutionError`	A Compose file names a variable or an `env_file` that prepare cannot resolve.
`UnsupportedComposeFieldError`	A key the sandbox Compose adapter does not start at all (`unsupported_compose_fields`), as opposed to a

Functions

Name	Description
`_bytes`	-
`_compose_scalar`	An environment value as Compose would emit it: booleans lower-case, `None` empty.
`_cpus`	-
`_devices`	GPUs requested by `deploy.resources.reservations.devices`: every entry must ask for the `gpu`
`_environment_mapping`	A service’s `environment` as a mapping. A bare `NAME` in the list form, or a null value in the mapping
`_memory`	-
`_normalize_restart`	-
`_service_networks`	A service’s `networks` as a mapping of network name to options; each must be declared (`default` is
`_unsupported_fields`	`(field, message)` per offending key of `document`, in document order: the top level first, then each
`check_compose_fields`	Give every M28 service field its defined meaning, or raise `ComposeFieldError` naming the field.
`normalize_deploy`	A service’s `deploy` block in the form the adapter reads.
`normalize_extra_hosts`	`extra_hosts` as a hostname-to-address mapping, from the list (`host:ip`, `host=ip`) or mapping form.
`normalize_hooks`	`post_start` entries as `command` (a string, run through the shell, or an argv list), optional `user`
`normalize_networks`	Top-level `networks` in the shape the adapter reads; every one maps onto the group’s single pod network.
`normalize_platform`	`os/arch[/variant]` as `os/arch` with Docker’s architecture aliases folded (`x86_64` is `amd64`).
`normalize_ports`	A service’s `ports` in long syntax, each entry checked the way the sandbox group runs it.
`normalize_pull_policy`	Compose `pull_policy` as `always`, `missing` or `never`.
`normalize_service_options`	`tmpfs`, `ulimits`, `sysctls` and `security_opt` in one shape each: a list of mounts, a mapping of
`service_gpus`	GPUs a service requests through `deploy.resources.reservations.devices` (0 when none).
`service_resources`	The sandbox resources a normalized service asks for: `cpu`, `memory_mib` and `gpu` when set.
`unsupported_compose_fields`	The refusals the sandbox Compose adapter raises for the keys of `document`: one message per offender in

Data

DEFAULT_PLATFORM

RESTART_POLICIES

SUPPORTED_COMPOSE_FIELDS

SUPPORTED_SERVICE_FIELDS

_SERVICE_NETWORK_KEYS

API

class nemo_gym.sandbox.compose_fields.ComposeFieldError(
    field: str,
    message: str
)

Bases: ValueError

A Compose service field prepare cannot run as written; field is the Compose key it names.

class nemo_gym.sandbox.compose_fields.ComposeSubstitutionError(
    name: str,
    message: str | None = None
)

Bases: ValueError

A Compose file names a variable or an env_file that prepare cannot resolve.

class nemo_gym.sandbox.compose_fields.UnsupportedComposeFieldError()

Bases: ComposeFieldError

A key the sandbox Compose adapter does not start at all (unsupported_compose_fields), as opposed to a supported field written in a way the group cannot run.

nemo_gym.sandbox.compose_fields._bytes(
    value: str | int
) -> int

nemo_gym.sandbox.compose_fields._compose_scalar(
    value: typing.Any
) -> str

An environment value as Compose would emit it: booleans lower-case, None empty.

nemo_gym.sandbox.compose_fields._cpus(
    value: typing.Any,
    where: str
) -> float

nemo_gym.sandbox.compose_fields._devices(
    devices: typing.Any,
    where: str
) -> int

GPUs requested by deploy.resources.reservations.devices: every entry must ask for the gpu capability with a numeric count or explicit device_ids; count: all has no meaning in a pod.

nemo_gym.sandbox.compose_fields._environment_mapping(
    service_name: str,
    environment: typing.Any,
    variables: dict[str, str]
) -> dict[str, str | None]

A service’s environment as a mapping. A bare NAME in the list form, or a null value in the mapping form, stands for the task variable of that name and is None when there is none: Compose then leaves the variable unset in the container, even where an env_file sets it. NAME= and NAME: '' are empty.

nemo_gym.sandbox.compose_fields._memory(
    value: typing.Any,
    where: str
) -> int

nemo_gym.sandbox.compose_fields._normalize_restart(
    service_name: str,
    value: typing.Any
) -> str

nemo_gym.sandbox.compose_fields._service_networks(
    service_name: str,
    value: typing.Any,
    declared: dict[str, typing.Any]
) -> dict[str, dict[str, typing.Any]]

A service’s networks as a mapping of network name to options; each must be declared (default is implicit). Only aliases has a meaning; static addresses and the rest raise.

nemo_gym.sandbox.compose_fields._unsupported_fields(
    document: typing.Any
) -> list[tuple[str, str]]

(field, message) per offending key of document, in document order: the top level first, then each service; field is the first unknown key of that level. Only mappings are inspected: a document or service that is not one fails the adapter’s shape checks instead.

nemo_gym.sandbox.compose_fields.check_compose_fields(
    document: dict,
    platform: str = DEFAULT_PLATFORM,
    provider: str | None = None,
    service_option_fields: frozenset[str] | None = None,
    gpu_provider: str | None = None,
    gpu_service_option_fields: frozenset[str] | None = None
) -> dict

Give every M28 service field its defined meaning, or raise ComposeFieldError naming the field.

The one field check: prepare runs it per task (nemo_gym.data.tasks.harbor.image_configs.check_task_compose_fields) and the adapter again at start. A key outside the allow-list (unsupported_compose_fields: a sidecar build:, privileged, logging, …) is an UnsupportedComposeFieldError before any value is read. Mutates and returns document, and applied to its own output returns it unchanged: the server normalizes once and the adapter again, from the written file. deploy is normalized (see normalize_deploy); restart is validated and kept (recorded by the adapter, the group restarts nothing); pull_policy becomes always, missing or never; platform must match platform, the one the deployment runs (see deployment_platform); networks (top level and per service) and extra_hosts take their mapping forms; post_start hooks are checked; ports take their long syntax, with the group’s endpoint rules applied (see normalize_ports); tmpfs, ulimits, sysctls and security_opt are normalized and, when provider is given, each must be in service_option_fields (the provider’s SERVICE_OPTION_FIELDS) or the error names the field and the provider. A sidecar that reserves GPUs through deploy.resources is checked against gpu_provider and gpu_service_option_fields instead when gpu_provider is given, since that is the provider it runs on (main follows the task, not its deploy block). Without a provider name the capability check is left to the adapter, which runs it against the live provider of each service before any sandbox.

nemo_gym.sandbox.compose_fields.normalize_deploy(
    service_name: str,
    deploy: typing.Any
) -> dict[str, typing.Any]

A service’s deploy block in the form the adapter reads.

resources.limits and resources.reservations carry cpus (a number) and memory (bytes); reservations.devices becomes gpus, the GPU count. replicas must be 1. restart_policy is kept as written and recorded, like restart. limits.pids and every other key name swarm scheduling and raise ComposeFieldError.

nemo_gym.sandbox.compose_fields.normalize_extra_hosts(
    service_name: str,
    value: typing.Any
) -> dict[str, str]

extra_hosts as a hostname-to-address mapping, from the list (host:ip, host=ip) or mapping form. host-gateway has no meaning in a pod and raises.

nemo_gym.sandbox.compose_fields.normalize_hooks(
    service_name: str,
    hooks: typing.Any
) -> list[dict[str, typing.Any]]

post_start entries as command (a string, run through the shell, or an argv list), optional user (a name, or a numeric id as an integer), working_dir and environment (a mapping). privileged raises: no hook escalates.

nemo_gym.sandbox.compose_fields.normalize_networks(
    document: dict
) -> dict[str, dict[str, typing.Any]]

Top-level networks in the shape the adapter reads; every one maps onto the group’s single pod network.

name, driver, external, attachable and labels are accepted and have no further meaning: services sharing a network reach each other by name, as on any Compose network. internal: true, ipam with a configuration, driver_opts and enable_ipv6: true ask for isolation or addressing the pod does not provide and raise ComposeFieldError naming them. Compose’s defaults spelled out (internal: false, enable_ipv6: false, an empty ipam block) ask for nothing and are dropped.

nemo_gym.sandbox.compose_fields.normalize_platform(
    value: str
) -> str

os/arch[/variant] as os/arch with Docker’s architecture aliases folded (x86_64 is amd64).

nemo_gym.sandbox.compose_fields.normalize_ports(
    service_name: str,
    value: typing.Any
) -> list[dict[str, typing.Any]]

A service’s ports in long syntax, each entry checked the way the sandbox group runs it.

A short entry ("8080", "8080:80", "127.0.0.1:8080:80/udp", a bare integer) takes the mapping Compose gives it (target, and published, host_ip, protocol when written), so prepare and the adapter read one shape. The group exposes a service’s ports as endpoints the provider allocates, so a fixed published port (anything but none or 0), a host_ip, a protocol other than TCP, a mode other than ingress, a port range and an unknown option raise ComposeFieldError naming the service: the refusals the adapter makes at seed, made at prepare, for the short and the long syntax alike.

nemo_gym.sandbox.compose_fields.normalize_pull_policy(
    service_name: str,
    value: typing.Any
) -> str

Compose pull_policy as always, missing or never.

if_not_present is missing; the age forms daily, weekly and every_<duration> are also missing, because a sandbox group holds no pull history. build raises: services do not build.

nemo_gym.sandbox.compose_fields.normalize_service_options(
    service_name: str,
    service: dict
) -> dict[str, typing.Any]

tmpfs, ulimits, sysctls and security_opt in one shape each: a list of mounts, a mapping of name to an integer or {soft, hard}, a mapping of key to string, and a list of options.

nemo_gym.sandbox.compose_fields.service_gpus(
    service: dict
) -> int

GPUs a service requests through deploy.resources.reservations.devices (0 when none).

Read from the normalized reservations.gpus alone: the field check calls this on every sidecar before resolve_compose has converted the legacy mem_limit/shm_size strings, so a CPU-only service with mem_limit: 512M must not be sized here.

nemo_gym.sandbox.compose_fields.service_resources(
    service: dict
) -> dict[str, typing.Any]

The sandbox resources a normalized service asks for: cpu, memory_mib and gpu when set.

deploy.resources.limits wins, then reservations, then the legacy cpus/mem_limit keys. GPUs come from reservations.devices only, as Compose defines them.

nemo_gym.sandbox.compose_fields.unsupported_compose_fields(
    document: typing.Any
) -> list[str]

The refusals the sandbox Compose adapter raises for the keys of document: one message per offender in document order (the top level first, then each service), empty when every key is one the adapter starts.

The key allow-list is the first step of check_compose_fields, the one field check prepare and the adapter run (a refused key is an UnsupportedComposeFieldError there). Pure and key-only, so prepare also screens the raw file of a task about to be built with it (nemo_gym.data.tasks.harbor.prepare), before interpolation gives the values their shapes.

nemo_gym.sandbox.compose_fields.DEFAULT_PLATFORM = 'linux/amd64'

nemo_gym.sandbox.compose_fields.RESTART_POLICIES = ('no', 'always', 'on-failure', 'unless-stopped')

nemo_gym.sandbox.compose_fields.SUPPORTED_COMPOSE_FIELDS = frozenset({'name', 'services', 'networks', 'volumes'})

nemo_gym.sandbox.compose_fields.SUPPORTED_SERVICE_FIELDS = frozenset({'image', 'entrypoint', 'command', 'environment', 'working_dir', 'user...

nemo_gym.sandbox.compose_fields._ARCHITECTURES = {'amd64': 'amd64', 'x86_64': 'amd64', 'x86-64': 'amd64', 'arm64': 'arm64', 'aarc...

nemo_gym.sandbox.compose_fields._DEPLOY_KEYS = frozenset({'resources', 'replicas', 'restart_policy'})

nemo_gym.sandbox.compose_fields._HOOK_KEYS = frozenset({'command', 'user', 'working_dir', 'environment'})

nemo_gym.sandbox.compose_fields._NETWORK_KEYS = frozenset({'name', 'driver', 'external', 'attachable', 'labels'})

nemo_gym.sandbox.compose_fields._PORT_KEYS = frozenset({'target', 'published', 'host_ip', 'protocol', 'mode'})

nemo_gym.sandbox.compose_fields._PULL_MISSING = ('missing', 'if_not_present', 'daily', 'weekly')

nemo_gym.sandbox.compose_fields._SERVICE_NETWORK_KEYS = frozenset({'aliases'})