> This page is for version 0.5.1. > For other versions, use one of these documentation indexes: > - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt > - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt > - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt > - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt > - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt > - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt > - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt > - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server. # nemo_gym.data.tasks.harbor.verifier_env Harbor's separate-verifier rules. Which environment the verifier runs in, the verifier table or a copy of the task environment, whether tests/ is a build context of its own, which top-level keys, sidecars and main-service keys a tests/docker-compose.yaml may declare (and which such files cannot be read at all), and the two refusals the loader raises when a separate verifier has no image and no definition. The loader and the build planner both read these rules. ## Module Contents ### Functions | Name | Description | | ------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | | [`_verifier_build_required`](#nemo_gym-data-tasks-harbor-verifier_env-_verifier_build_required) | `root` is the dataset folder whose `built-images.json` the loader read (see `_build_required`). | | [`_verifier_compose_services`](#nemo_gym-data-tasks-harbor-verifier_env-_verifier_compose_services) | The `services` mapping of the task's `tests/docker-compose.yaml`; `{}` without the file, or for one that | | [`_verifier_definition_required`](#nemo_gym-data-tasks-harbor-verifier_env-_verifier_definition_required) | Harbor's refusal of a separate verifier environment with nothing to run: its environment dir is always | | [`effective_verifier_environment`](#nemo_gym-data-tasks-harbor-verifier_env-effective_verifier_environment) | The environment a separate verifier runs in, as Harbor resolves it, or `None` in shared mode. | | [`step_verifier_build_context`](#nemo_gym-data-tasks-harbor-verifier_env-step_verifier_build_context) | The folder Harbor starts a step's separate verifier environment from (`_verifier_env_build_context` with a | | [`step_verifier_environment`](#nemo_gym-data-tasks-harbor-verifier_env-step_verifier_environment) | The environment a step's separate verifier runs in, as Harbor's `resolve_effective_verifier_env_config` | | [`step_verifier_mode`](#nemo_gym-data-tasks-harbor-verifier_env-step_verifier_mode) | `"separate"` or `"shared"` for one step, as Harbor's `resolve_step_verifier_mode` (verifier_mode.py at | | [`verifier_build_context`](#nemo_gym-data-tasks-harbor-verifier_env-verifier_build_context) | The folder Harbor builds the separate verifier's image from, `tests/`, when it builds one; else `None`. | | [`verifier_build_environment`](#nemo_gym-data-tasks-harbor-verifier_env-verifier_build_environment) | The effective verifier environment whose settings Harbor's build of `tests/Dockerfile` sees, or `None` | | [`verifier_compose_main_overrides`](#nemo_gym-data-tasks-harbor-verifier_env-verifier_compose_main_overrides) | The keys of `main` in the task's `tests/docker-compose.yaml` other than `VERIFIER_MAIN_KEYS` (other | | [`verifier_compose_refusal`](#nemo_gym-data-tasks-harbor-verifier_env-verifier_compose_refusal) | Why the task's `tests/docker-compose.yaml` is not the file `verifier_compose_sidecars` and | | [`verifier_compose_sidecars`](#nemo_gym-data-tasks-harbor-verifier_env-verifier_compose_sidecars) | The services other than `main` that the task's `tests/docker-compose.yaml` declares, in file order; `[]` | ### Data [`BUILD_PULL_POLICY`](#nemo_gym-data-tasks-harbor-verifier_env-BUILD_PULL_POLICY) [`COMPOSE_EXTENSION_PREFIX`](#nemo_gym-data-tasks-harbor-verifier_env-COMPOSE_EXTENSION_PREFIX) [`PREBUILT_VERIFIER_MAIN_KEYS`](#nemo_gym-data-tasks-harbor-verifier_env-PREBUILT_VERIFIER_MAIN_KEYS) [`TESTS_COMPOSE_FILE`](#nemo_gym-data-tasks-harbor-verifier_env-TESTS_COMPOSE_FILE) [`TESTS_DOCKERFILE`](#nemo_gym-data-tasks-harbor-verifier_env-TESTS_DOCKERFILE) [`VERIFIER_COMPOSE_TOP_LEVEL_KEYS`](#nemo_gym-data-tasks-harbor-verifier_env-VERIFIER_COMPOSE_TOP_LEVEL_KEYS) [`VERIFIER_MAIN_KEYS`](#nemo_gym-data-tasks-harbor-verifier_env-VERIFIER_MAIN_KEYS) ### API ```python nemo_gym.data.tasks.harbor.verifier_env._verifier_build_required( task_folder: pathlib.Path, root: pathlib.Path ) -> nemo_gym.data.tasks.harbor.errors.HarborTaskError ``` `root` is the dataset folder whose `built-images.json` the loader read (see `_build_required`). ```python nemo_gym.data.tasks.harbor.verifier_env._verifier_compose_services( task_folder: pathlib.Path ) -> dict[typing.Any, typing.Any] ``` The `services` mapping of the task's `tests/docker-compose.yaml`; `{}` without the file, or for one that does not parse or has no services mapping (a file the loader refuses first, `verifier_compose_refusal`). ```python nemo_gym.data.tasks.harbor.verifier_env._verifier_definition_required( task_folder: pathlib.Path ) -> nemo_gym.data.tasks.harbor.errors.HarborTaskError ``` Harbor's refusal of a separate verifier environment with nothing to run: its environment dir is always `tests/` (`_verifier_env_build_context` in trial.py), checked by `require_agent_environment_definition`. ```python nemo_gym.data.tasks.harbor.verifier_env.effective_verifier_environment( config: nemo_gym.data.tasks.harbor.models.HarborTaskConfig ) -> nemo_gym.data.tasks.harbor.models.HarborEnvironment | None ``` The environment a separate verifier runs in, as Harbor resolves it, or `None` in shared mode. Harbor's `resolve_effective_verifier_env_config` (verifier_mode.py at v0.22.0) for a single-step task: `[verifier.environment]` when the task declares one, else a copy of `[environment]`, so a task that only says `environment_mode = "separate"` grades in an environment shaped like the agent's, `docker_image` included. ```python nemo_gym.data.tasks.harbor.verifier_env.step_verifier_build_context( task_folder: pathlib.Path, step: nemo_gym.data.tasks.harbor.models.HarborTaskStep ) -> pathlib.Path ``` The folder Harbor starts a step's separate verifier environment from (`_verifier_env_build_context` with a step in trial.py at v0.22.0): `steps//tests/` when that folder exists, else the task's `tests/`. ```python nemo_gym.data.tasks.harbor.verifier_env.step_verifier_environment( config: nemo_gym.data.tasks.harbor.models.HarborTaskConfig, step: nemo_gym.data.tasks.harbor.models.HarborTaskStep ) -> nemo_gym.data.tasks.harbor.models.HarborEnvironment | None ``` The environment a step's separate verifier runs in, as Harbor's `resolve_effective_verifier_env_config` resolves it with a step: `[steps.verifier.environment]`, else `[verifier.environment]`, else a copy of `[environment]`; `None` when the step verifies in shared mode. ```python nemo_gym.data.tasks.harbor.verifier_env.step_verifier_mode( config: nemo_gym.data.tasks.harbor.models.HarborTaskConfig, step: nemo_gym.data.tasks.harbor.models.HarborTaskStep ) -> str ``` `"separate"` or `"shared"` for one step, as Harbor's `resolve_step_verifier_mode` (verifier_mode.py at v0.22.0) resolves it: the step's explicit `[steps.verifier].environment_mode`, else `separate` when the step declares a `[steps.verifier.environment]`, else the task's own mode (`HarborTaskConfig.is_shared_verifier`). Steps of one task may resolve differently. ```python nemo_gym.data.tasks.harbor.verifier_env.verifier_build_context( task_folder: pathlib.Path, config: nemo_gym.data.tasks.harbor.models.HarborTaskConfig ) -> pathlib.Path | None ``` The folder Harbor builds the separate verifier's image from, `tests/`, when it builds one; else `None`. Harbor starts the separate verifier's environment with `tests/` as its environment dir (`_verifier_env_build_context` in trial.py at v0.22.0) and the effective verifier environment as its config (`effective_verifier_environment`). Its Docker environment then uses the config's `docker_image` when there is one and otherwise runs `docker compose build` on `tests/Dockerfile` (`should_use_prebuilt_docker_image`, `start` in docker.py), whatever that Dockerfile holds: a `FROM` plus `RUN` file is built, not overlaid. So: shared mode, a declared or inherited `docker_image`, or no `tests/Dockerfile` means no build. Without the file and without an image Harbor refuses the task (`require_agent_environment_definition` on `tests/`, which is always its verifier environment dir: there is no fallback to `environment/`), and so does `load_task`. Prepare plans one build entry per returned context (`nemo_gym.data.tasks.harbor.image_build.plan_builds`) and the loader reads its record. A multi-step task whose steps verify separately builds `tests/` for the steps that declare no image and start from it (`verifier_build_environment`), whatever the task's own mode. ```python nemo_gym.data.tasks.harbor.verifier_env.verifier_build_environment( task_folder: pathlib.Path, config: nemo_gym.data.tasks.harbor.models.HarborTaskConfig ) -> nemo_gym.data.tasks.harbor.models.HarborEnvironment | None ``` The effective verifier environment whose settings Harbor's build of `tests/Dockerfile` sees, or `None` when nothing builds from `tests/`: the task's own (`effective_verifier_environment`) when it declares no `docker_image`, else the first step's whose separate verifier declares none and starts from `tests/` (`step_verifier_build_context`; a step with a `tests/` folder of its own starts from that folder, which this server does not build, and the loader refuses such a step). Steps that would build `tests/` under different environments share the first one's settings here, where Harbor builds once per step. ```python nemo_gym.data.tasks.harbor.verifier_env.verifier_compose_main_overrides( task_folder: pathlib.Path, prebuilt: bool ) -> list[str] ``` The keys of `main` in the task's `tests/docker-compose.yaml` other than `VERIFIER_MAIN_KEYS` (other than `PREBUILT_VERIFIER_MAIN_KEYS` when `prebuilt`: the effective verifier environment names a `docker_image`, so nothing builds from `tests/`, and a `pull_policy` of `build` counts too), in file order; `[]` without the file or without a `main` mapping. Harbor stacks that file on its verifier template for the build and for `up` (`_docker_compose_paths` in docker.py at v0.22.0), so `environment`, `env_file`, `volumes`, `command` and every other runtime key of `main` reach the verifier's container: a `test.sh` that reads `CHECK_MODE` from `main.environment` sees it there, an `image` beside a prebuilt verifier replaces the declared image Harbor's template names (`${PREBUILT_IMAGE_NAME}`), and a `build` section there has Harbor's `up` build `tests/Dockerfile` under the declared name, always with `pull_policy: build` (see `VERIFIER_MAIN_KEYS`). This server starts the separate verifier as a single sandbox from the effective verifier environment of `task.toml` and the verifier image alone, and reads the file for the build section and pinned `image` of a `main` it builds only, so such a task would grade differently from Harbor without a word. It is refused instead, like one with verifier sidecars: by `load_task` and, before any build, by `ImageBuilder.build_many`. ```python nemo_gym.data.tasks.harbor.verifier_env.verifier_compose_refusal( task_folder: pathlib.Path ) -> str | None ``` Why the task's `tests/docker-compose.yaml` is not the file `verifier_compose_sidecars` and `verifier_compose_main_overrides` read, as the loader's message, or `None` when it is (or there is none). Those readers see the file's own `services` mapping and nothing else, so what they cannot see must not reach Harbor's verifier either. Harbor stacks the whole file on its verifier template (`_docker_compose_paths` in docker.py at v0.22.0) and `compose up` applies every top-level key of it to the verifier's project, so only the keys this server reads or that cannot shape the verifier's container pass (`VERIFIER_COMPOSE_TOP_LEVEL_KEYS` and `x-` extension fields) and any other is refused by name. A top-level `include` merges the included files' services into the project before the stacking, so `main.environment` or a `build` section written in an included file reaches Harbor's verifier container as if written here, while this server reads `services: {}` and grades in the declared image without a word (`validate_task_compose` refuses `include` for a Dockerfile build the same way; this is the loader's refusal, for every separate verifier, prebuilt ones, which that validator never sees, included). A top-level `networks` reconfigures the project's `default` network, the one Compose attaches `main` to because neither of Harbor's templates gives it a `networks` or `network_mode` key and, under Harbor's default `public` policy, no egress overlay does either: `default: {internal: true}` grades Harbor's verifier without egress (`driver_opts`, `ipam`, `enable_ipv6` likewise) while this server's single sandbox grades with it. Top-level `volumes`, `configs` and `secrets` are inert until a service references them, and a reference on `main` is a key `verifier_compose_main_overrides` refuses, but the class is closed rather than judged key by key, since each is a project setting the readers never see. A file that is not valid YAML or not a mapping, or whose `services` is not one, fails Harbor's `compose up` (and Harbor's own reader of the file, `_egress_controlled_service_names` in docker.py, parses it with `yaml.safe_load`), so the trial errors under Harbor where the readers above would see `{}`; refused as `validate_task_compose` refuses it. An empty file is an empty mapping, as it is there. ```python nemo_gym.data.tasks.harbor.verifier_env.verifier_compose_sidecars( task_folder: pathlib.Path ) -> list[str] ``` The services other than `main` that the task's `tests/docker-compose.yaml` declares, in file order; `[]` without the file, or for one that does not parse or has no services mapping (`verifier_compose_refusal` names such a file, and a top-level key other than `services`, first). Harbor's separate verifier is a whole Docker environment over `tests/`: it stacks that Compose file on its build or prebuilt template (`_docker_compose_paths` in docker.py at v0.22.0), builds every service with a `build` section and starts them all with `up`, image or built, prebuilt `main` or not. This server starts the separate verifier as a single sandbox, so a task that declares any such service is refused: by `load_task` and, before any build, by `ImageBuilder.build_many`. A `main`-only file (cadir's and arc-crm-6's shape) is read for `main`'s build section, and its `image` when prepare builds the verifier; a runtime setting on `main` (every key of arc-crm-6's file: `network_mode`, `pids_limit`, `security_opt`, `cap_drop`) is refused the same way (`verifier_compose_main_overrides`). ```python nemo_gym.data.tasks.harbor.verifier_env.BUILD_PULL_POLICY = 'build' ``` ```python nemo_gym.data.tasks.harbor.verifier_env.COMPOSE_EXTENSION_PREFIX = 'x-' ``` ```python nemo_gym.data.tasks.harbor.verifier_env.PREBUILT_VERIFIER_MAIN_KEYS = frozenset({'pull_policy'}) ``` ```python nemo_gym.data.tasks.harbor.verifier_env.TESTS_COMPOSE_FILE = Path('tests') / 'docker-compose.yaml' ``` ```python nemo_gym.data.tasks.harbor.verifier_env.TESTS_DOCKERFILE = Path('tests') / 'Dockerfile' ``` ```python nemo_gym.data.tasks.harbor.verifier_env.VERIFIER_COMPOSE_TOP_LEVEL_KEYS = frozenset({'services', 'name', 'version'}) ``` ```python nemo_gym.data.tasks.harbor.verifier_env.VERIFIER_MAIN_KEYS = frozenset({'image', 'build', 'pull_policy'}) ```