> This page is for version 0.5.1. > For other versions, use one of these documentation indexes: > - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt > - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt > - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt > - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt > - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt > - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt > - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt > - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server. # nemo_gym.data.tasks.harbor.dockerfile Recognize Dockerfiles Gym can run without a build, and resolve them as Docker would. Two shapes are recognized: * *Pull mode*: `FROM ` plus `WORKDIR`, `ENV`, `USER` or `LABEL` lines. Nothing changes the filesystem, so Gym runs the base image directly and applies the recorded settings at sandbox creation (`base_image_only`). * *Overlay mode*: the same, plus `RUN` lines. Gym pulls the base image and runs the `RUN` lines inside the sandbox at seed, each with the `WORKDIR`, `ENV` and `USER` in effect at that point of the Dockerfile (`overlay_image`). This is the shape of SWE-bench style tasks: a published environment image plus setup commands. `ENV`, `WORKDIR` and `USER` are resolved the way `docker build` resolves them: the base image's own configuration (`BaseImageConfig`, recorded from the registry at prepare) is the starting state, `$NAME` and `${NAME}` expand against the accumulated environment (not inside single quotes; `\$` is a literal dollar), a relative `WORKDIR` joins onto the previous one. The resolved values are literal: nothing reaches the sandbox with a live `$` in it. A Dockerfile whose resolution depends on the base image (a `RUN` line, a reference to a variable the Dockerfile did not set, a relative first `WORKDIR`) raises `ImageConfigRequired` when no configuration is given. An empty `CMD []` is accepted and ignored. An empty `ENTRYPOINT []` is accepted and recorded (`entrypoint_cleared`): the sandbox provider starts the image with its own keepalive either way, and the Harbor environment server runs the base image's recorded `Entrypoint` beside it unless the Dockerfile cleared it. Anything else (`COPY`, `ADD`, multi-stage `FROM`, `ARG`-templated images, heredoc `RUN`, `RUN --mount`, a real `ENTRYPOINT` or `CMD`, `HEALTHCHECK`, `EXPOSE`, `VOLUME`, `SHELL`, `STOPSIGNAL`, a `FROM` flag such as `--platform`, an `# escape=` parser directive) needs a real build, which is not part of this loader yet; `DockerfileNeedsBuild` names the first instruction that requires it. ## Module Contents ### Classes | Name | Description | | ------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | | [`BaseImage`](#nemo_gym-data-tasks-harbor-dockerfile-BaseImage) | A Dockerfile that reduces to a base image plus creation-time settings. | | [`BaseImageConfig`](#nemo_gym-data-tasks-harbor-dockerfile-BaseImageConfig) | What a base image's OCI configuration sets before its Dockerfile's first instruction. | | [`Dockerfile`](#nemo_gym-data-tasks-harbor-dockerfile-Dockerfile) | The parsed instructions of a pull-mode or overlay-mode Dockerfile, before resolution. | | [`DockerfileNeedsBuild`](#nemo_gym-data-tasks-harbor-dockerfile-DockerfileNeedsBuild) | The Dockerfile holds an instruction this loader cannot apply without a build. | | [`ImageConfigRequired`](#nemo_gym-data-tasks-harbor-dockerfile-ImageConfigRequired) | Resolving the Dockerfile needs the base image's recorded configuration, which was not given. | | [`OverlayImage`](#nemo_gym-data-tasks-harbor-dockerfile-OverlayImage) | A base image plus the `RUN` lines to apply at seed, in Dockerfile order. | | [`OverlayRun`](#nemo_gym-data-tasks-harbor-dockerfile-OverlayRun) | One `RUN` line with the Dockerfile state in effect when it runs, resolved to literals. | | [`OverlayWorkdir`](#nemo_gym-data-tasks-harbor-dockerfile-OverlayWorkdir) | One `WORKDIR` instruction: the directory to create, owned by the user in effect at it. | | [`_Expander`](#nemo_gym-data-tasks-harbor-dockerfile-_Expander) | Docker's shell-word processing: quote removal, escapes and `$` expansion against `env`. | ### Functions | Name | Description | | --------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | [`_closing_brace`](#nemo_gym-data-tasks-harbor-dockerfile-_closing_brace) | Index of the `}` closing the `{` that opens `text`, skipping nested `${...}`; -1 when unterminated. | | [`_env_pairs`](#nemo_gym-data-tasks-harbor-dockerfile-_env_pairs) | `(key, raw value)` pairs of an `ENV` line; values keep their quotes for the expander. | | [`_json_array`](#nemo_gym-data-tasks-harbor-dockerfile-_json_array) | The JSON array an exec-form instruction holds, or `None` when the arguments are shell form. | | [`_logical_lines`](#nemo_gym-data-tasks-harbor-dockerfile-_logical_lines) | Join continuation lines and drop comments and blanks, as Docker's parser does. | | [`_reject_escape_directive`](#nemo_gym-data-tasks-harbor-dockerfile-_reject_escape_directive) | Refuse a Dockerfile whose leading comment block sets `# escape=`. | | [`_run_command`](#nemo_gym-data-tasks-harbor-dockerfile-_run_command) | The validated arguments of a `RUN` line: a shell-form line, or the exec-form JSON array as written. | | [`base_image_only`](#nemo_gym-data-tasks-harbor-dockerfile-base_image_only) | Return the base image and settings, or `None` when the file has `RUN` lines or needs a build. | | [`dockerfile_has_heredoc`](#nemo_gym-data-tasks-harbor-dockerfile-dockerfile_has_heredoc) | Whether any `RUN`, `COPY` or `ADD` instruction of a Dockerfile opens a heredoc. | | [`has_heredoc`](#nemo_gym-data-tasks-harbor-dockerfile-has_heredoc) | Whether `command` opens a Dockerfile heredoc, judged word by word as BuildKit does. | | [`overlay_image`](#nemo_gym-data-tasks-harbor-dockerfile-overlay_image) | The base image, settings and `RUN` lines, or `None` when the file needs a build. | | [`parse_dockerfile`](#nemo_gym-data-tasks-harbor-dockerfile-parse_dockerfile) | Read a pull-mode or overlay-mode Dockerfile; raise `DockerfileNeedsBuild` for anything else. | | [`resolve_dockerfile`](#nemo_gym-data-tasks-harbor-dockerfile-resolve_dockerfile) | Apply the instructions in order from the base image's state, as `docker build` would. | | [`split_words`](#nemo_gym-data-tasks-harbor-dockerfile-split_words) | Split on unquoted blanks, keeping quotes and escapes in the words (Docker's `parseWords`). | ### Data [`DEFAULT_PATH`](#nemo_gym-data-tasks-harbor-dockerfile-DEFAULT_PATH) [`DEFAULT_SHELL`](#nemo_gym-data-tasks-harbor-dockerfile-DEFAULT_SHELL) [`RECORD_VERSION`](#nemo_gym-data-tasks-harbor-dockerfile-RECORD_VERSION) [`_BLANKS`](#nemo_gym-data-tasks-harbor-dockerfile-_BLANKS) [`_CONTINUATION`](#nemo_gym-data-tasks-harbor-dockerfile-_CONTINUATION) [`_HEREDOC_WORD`](#nemo_gym-data-tasks-harbor-dockerfile-_HEREDOC_WORD) [`_NAME`](#nemo_gym-data-tasks-harbor-dockerfile-_NAME) ### API ```python class nemo_gym.data.tasks.harbor.dockerfile.BaseImage( image: str, workdir: str | None = None, env: dict[str, str] = dict(), user: str | None = None, entrypoint_cleared: bool = False ) ``` Dataclass A Dockerfile that reduces to a base image plus creation-time settings. `env` holds the keys the Dockerfile's `ENV` lines set, with their resolved final values; `workdir` and `user` are the resolved final values (`None` when neither the Dockerfile nor a given base image configuration sets them). `entrypoint_cleared` is whether the Dockerfile wrote `ENTRYPOINT []`, so the base image's entrypoint must not run. **`entrypoint_cleared`** `bool = False` --- **`env`** `dict[str, str] = field(default_factory=dict)` --- **`image`** `str` --- **`user`** `str | None = None` --- **`workdir`** `str | None = None` --- ```python class nemo_gym.data.tasks.harbor.dockerfile.BaseImageConfig( env: dict[str, str] = dict(), workdir: str | None = None, user: str | None = None, entrypoint: tuple[str, ...] | None = None, healthcheck: dict[str, typing.Any] | None = None, shell: tuple[str, ...] = DEFAULT_SHELL ) ``` Dataclass What a base image's OCI configuration sets before its Dockerfile's first instruction. `env` is the image's `Env` as a mapping; `workdir` and `user` are `WorkingDir` and `User` (`None` when the image sets none, meaning `/` and root); `shell` is the image's `Shell`, which runs its shell-form `RUN` lines (Docker's `/bin/sh -c` when unset). **`entrypoint`** `tuple[str, ...] | None = None` --- **`env`** `dict[str, str] = field(default_factory=dict)` --- **`healthcheck`** `dict[str, Any] | None = None` --- **`shell`** `tuple[str, ...] = DEFAULT_SHELL` --- **`user`** `str | None = None` --- **`workdir`** `str | None = None` --- ```python nemo_gym.data.tasks.harbor.dockerfile.BaseImageConfig.from_record( record: collections.abc.Mapping[str, typing.Any] ) -> nemo_gym.data.tasks.harbor.dockerfile.BaseImageConfig ``` classmethod Read a `compose-images.json` entry (see `nemo_gym.data.tasks.harbor.image_configs`). Raises `ValueError` for an entry written before `RECORD_VERSION`: it lacks fields this resolver reads (an absent `Env` would read as a known, empty environment), so the dataset must be prepared again to re-record it. ```python class nemo_gym.data.tasks.harbor.dockerfile.Dockerfile( image: str, instructions: tuple[tuple[str, str], ...] = (), entrypoint_cleared: bool = False ) ``` Dataclass The parsed instructions of a pull-mode or overlay-mode Dockerfile, before resolution. **`entrypoint_cleared`** `bool = False` --- **`has_runs`** `bool` --- **`image`** `str` --- **`instructions`** `tuple[tuple[str, str], ...] = ()` --- ```python class nemo_gym.data.tasks.harbor.dockerfile.DockerfileNeedsBuild() ``` **Bases:** `ValueError` The Dockerfile holds an instruction this loader cannot apply without a build. ```python class nemo_gym.data.tasks.harbor.dockerfile.ImageConfigRequired() ``` **Bases:** `ValueError` Resolving the Dockerfile needs the base image's recorded configuration, which was not given. ```python class nemo_gym.data.tasks.harbor.dockerfile.OverlayImage( image: str, workdir: str | None = None, env: dict[str, str] = dict(), user: str | None = None, entrypoint_cleared: bool = False, runs: tuple[nemo_gym.data.tasks.harbor.dockerfile.OverlayRun, ...] = (), workdirs: tuple[nemo_gym.data.tasks.harbor.dockerfile.OverlayWorkdir, ...] = () ) ``` Dataclass **Bases:** [BaseImage](#nemo_gym-data-tasks-harbor-dockerfile-BaseImage) A base image plus the `RUN` lines to apply at seed, in Dockerfile order. `workdirs` are the `WORKDIR` instructions, in order, each with the user in effect at it and its place among the runs (see `OverlayWorkdir`). **`runs`** `tuple[OverlayRun, ...] = ()` --- **`workdirs`** `tuple[OverlayWorkdir, ...] = ()` --- ```python class nemo_gym.data.tasks.harbor.dockerfile.OverlayRun( command: str, workdir: str = '/', env: dict[str, str] = dict(), user: str | None = None, shell: tuple[str, ...] | None = DEFAULT_SHELL ) ``` Dataclass One `RUN` line with the Dockerfile state in effect when it runs, resolved to literals. `command` is the command text: a shell-form line as written, or an exec-form argument vector joined with shell quoting. `shell` runs it: the image's `Shell` in effect for a shell-form line (Docker's `/bin/sh -c` unless the image sets one), `None` for an exec-form line, which runs its arguments directly, without a shell. `env` is the whole environment the line runs with (the base image's `Env` plus the `ENV` lines so far), `workdir` the directory it runs in and `user` who runs it (`None` is root). **`command`** `str` --- **`env`** `dict[str, str] = field(default_factory=dict)` --- **`shell`** `tuple[str, ...] | None = DEFAULT_SHELL` --- **`user`** `str | None = None` --- **`workdir`** `str = '/'` --- ```python class nemo_gym.data.tasks.harbor.dockerfile.OverlayWorkdir( path: str, user: str | None = None, before: int = 0 ) ``` Dataclass One `WORKDIR` instruction: the directory to create, owned by the user in effect at it. `before` is how many `RUN` lines precede the instruction, which places it among `OverlayImage.runs`: the directory is made before run `before` (counting from 0), or after the last run when no `RUN` follows it. Docker creates every `WORKDIR` whether or not a `RUN` uses it, so the finished image has them all. **`before`** `int = 0` --- **`path`** `str` --- **`user`** `str | None = None` --- ```python class nemo_gym.data.tasks.harbor.dockerfile._Expander( env: collections.abc.Mapping[str, str], known: bool ) ``` Docker's shell-word processing: quote removal, escapes and `$` expansion against `env`. With `known=False` the base image's environment is unknown, so a reference to a variable the Dockerfile did not set itself raises `ImageConfigRequired` instead of silently expanding to nothing. ```python nemo_gym.data.tasks.harbor.dockerfile._Expander._dollar( word: str, index: int ) -> tuple[str, int] ``` Expand the `$` at `index`; return the text and the index after the expression. ```python nemo_gym.data.tasks.harbor.dockerfile._Expander._lookup( name: str ) -> str | None ``` ```python nemo_gym.data.tasks.harbor.dockerfile._Expander.expand( word: str ) -> str ``` ```python nemo_gym.data.tasks.harbor.dockerfile._closing_brace( text: str ) -> int ``` Index of the `}` closing the `{` that opens `text`, skipping nested `${...}`; -1 when unterminated. ```python nemo_gym.data.tasks.harbor.dockerfile._env_pairs( arguments: str ) -> list[tuple[str, str]] ``` `(key, raw value)` pairs of an `ENV` line; values keep their quotes for the expander. ```python nemo_gym.data.tasks.harbor.dockerfile._json_array( arguments: str ) -> list | None ``` The JSON array an exec-form instruction holds, or `None` when the arguments are shell form. ```python nemo_gym.data.tasks.harbor.dockerfile._logical_lines( text: str ) -> list[str] ``` Join continuation lines and drop comments and blanks, as Docker's parser does. A trailing backslash is removed together with the newline and the next line is appended verbatim (no separator is inserted). Comment and blank lines are dropped even inside a continuation. Only the first line of an instruction has its leading blanks removed. ```python nemo_gym.data.tasks.harbor.dockerfile._reject_escape_directive( text: str ) -> None ``` Refuse a Dockerfile whose leading comment block sets `# escape=`. Docker reads parser directives only in the comments before the first instruction, and this loader joins continuation lines with a backslash alone (`_logical_lines`), so a file that switches the escape character would be read with the wrong line structure (`RUN echo foo \` would swallow the next instruction). A build honours the directive; a `# escape=` after an instruction is a plain comment, as it is for Docker. `# syntax=` and `# check=` do not change how lines are read. ```python nemo_gym.data.tasks.harbor.dockerfile._run_command( arguments: str ) -> str ``` The validated arguments of a `RUN` line: a shell-form line, or the exec-form JSON array as written. ```python nemo_gym.data.tasks.harbor.dockerfile.base_image_only( text: str, base: nemo_gym.data.tasks.harbor.dockerfile.BaseImageConfig | None = None ) -> nemo_gym.data.tasks.harbor.dockerfile.BaseImage | None ``` Return the base image and settings, or `None` when the file has `RUN` lines or needs a build. ```python nemo_gym.data.tasks.harbor.dockerfile.dockerfile_has_heredoc( text: str ) -> bool ``` Whether any `RUN`, `COPY` or `ADD` instruction of a Dockerfile opens a heredoc. Each instruction's arguments are judged as `has_heredoc` judges one line, word by word as BuildKit does, so `RUN cat > /x << 'EOF'` (a space before the delimiter, which BuildKit does not read as a heredoc either) is not one. The body lines of a heredoc are not instructions, but they are only ever reached once the opening line has already answered. ```python nemo_gym.data.tasks.harbor.dockerfile.has_heredoc( command: str ) -> bool ``` Whether `command` opens a Dockerfile heredoc, judged word by word as BuildKit does. ```python nemo_gym.data.tasks.harbor.dockerfile.overlay_image( text: str, base: nemo_gym.data.tasks.harbor.dockerfile.BaseImageConfig | None = None ) -> nemo_gym.data.tasks.harbor.dockerfile.OverlayImage | None ``` The base image, settings and `RUN` lines, or `None` when the file needs a build. Raises `ImageConfigRequired` when the file depends on a base image configuration and none is given. ```python nemo_gym.data.tasks.harbor.dockerfile.parse_dockerfile( text: str ) -> nemo_gym.data.tasks.harbor.dockerfile.Dockerfile ``` Read a pull-mode or overlay-mode Dockerfile; raise `DockerfileNeedsBuild` for anything else. ```python nemo_gym.data.tasks.harbor.dockerfile.resolve_dockerfile( dockerfile: nemo_gym.data.tasks.harbor.dockerfile.Dockerfile, base: nemo_gym.data.tasks.harbor.dockerfile.BaseImageConfig | None = None ) -> nemo_gym.data.tasks.harbor.dockerfile.OverlayImage ``` Apply the instructions in order from the base image's state, as `docker build` would. With `base` given, the starting environment is its `Env` (plus Docker's default `PATH` when it has none), the working directory its `WorkingDir` or `/` and the user its `User`. Without it, only a Dockerfile whose values do not depend on the base image resolves; anything else raises `ImageConfigRequired`. ```python nemo_gym.data.tasks.harbor.dockerfile.split_words( text: str ) -> list[str] ``` Split on unquoted blanks, keeping quotes and escapes in the words (Docker's `parseWords`). ```python nemo_gym.data.tasks.harbor.dockerfile.DEFAULT_PATH = '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' ``` ```python nemo_gym.data.tasks.harbor.dockerfile.DEFAULT_SHELL = ('/bin/sh', '-c') ``` ```python nemo_gym.data.tasks.harbor.dockerfile.RECORD_VERSION = 2 ``` ```python nemo_gym.data.tasks.harbor.dockerfile._BLANKS = re.compile('[ \\t\\v\\f\\r]+') ``` ```python nemo_gym.data.tasks.harbor.dockerfile._CONTINUATION = re.compile('\\\\[ \\t]*$') ``` ```python nemo_gym.data.tasks.harbor.dockerfile._HEREDOC_WORD = re.compile('^<<-?(?:\\w+|\\"[^\\"]+\\"|\'[^\']+\')$') ``` ```python nemo_gym.data.tasks.harbor.dockerfile._NAME = re.compile('[A-Za-z_][A-Za-z0-9_]*') ```