> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo/gym/v0.3.0/nemo-gym/nemo_gym/sandbox/providers/apptainer/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server. # nemo_gym.sandbox.providers.apptainer Apptainer provider package. ## Submodules * **[`nemo_gym.sandbox.providers.apptainer.provider`](/nemo/gym/nemo-gym/nemo_gym/sandbox/providers/apptainer/provider)** ## Package Contents ### Classes | Name | Description | | --------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | | [`ApptainerCreateConfig`](#nemo_gym-sandbox-providers-apptainer-provider-ApptainerCreateConfig) | Settings for creating an Apptainer sandbox instance. | | [`ApptainerCreateError`](#nemo_gym-sandbox-providers-apptainer-provider-ApptainerCreateError) | Raised when Apptainer cannot create a sandbox. | | [`ApptainerCreateVerificationError`](#nemo_gym-sandbox-providers-apptainer-provider-ApptainerCreateVerificationError) | Raised when a newly-created sandbox cannot execute a probe command. | | [`ApptainerExecConfig`](#nemo_gym-sandbox-providers-apptainer-provider-ApptainerExecConfig) | Settings for running commands inside an Apptainer sandbox. | | [`ApptainerProbeConfig`](#nemo_gym-sandbox-providers-apptainer-provider-ApptainerProbeConfig) | Post-create probe settings: a test command confirming the sandbox is usable. | | [`ApptainerProvider`](#nemo_gym-sandbox-providers-apptainer-provider-ApptainerProvider) | Sandbox provider backed by the local Apptainer CLI. | ### API ```python class nemo_gym.sandbox.providers.apptainer.ApptainerCreateConfig( mount_point: str = DEFAULT_MOUNT_POINT, start_timeout_s: float | None = 600, extra_start_args: list[str] = list(), apply_resource_limits: bool = True ) ``` Dataclass Settings for creating an Apptainer sandbox instance. **`apply_resource_limits`** `bool = True` --- **`extra_start_args`** `list[str] = field(default_factory=list)` --- **`mount_point`** `str = DEFAULT_MOUNT_POINT` --- **`start_timeout_s`** `float | None = 600` --- ```python nemo_gym.sandbox.providers.apptainer.ApptainerCreateConfig.__post_init__() -> None ``` ```python class nemo_gym.sandbox.providers.apptainer.ApptainerCreateError() ``` **Bases:** [SandboxCreateError](/nemo/gym/nemo-gym/nemo_gym/sandbox/providers/base#nemo_gym-sandbox-providers-base-SandboxCreateError) Raised when Apptainer cannot create a sandbox. ```python class nemo_gym.sandbox.providers.apptainer.ApptainerCreateVerificationError() ``` **Bases:** [SandboxCreateVerificationError](/nemo/gym/nemo-gym/nemo_gym/sandbox/providers/base#nemo_gym-sandbox-providers-base-SandboxCreateVerificationError) Raised when a newly-created sandbox cannot execute a probe command. ```python class nemo_gym.sandbox.providers.apptainer.ApptainerExecConfig( default_timeout_s: float | None = 180, timeout_grace_s: float = 15, fakeroot_for_root: bool = True, default_binds: list[str] = list(), extra_exec_args: list[str] = list(), concurrency: int = 32 ) ``` Dataclass Settings for running commands inside an Apptainer sandbox. **`concurrency`** `int = 32` --- **`default_binds`** `list[str] = field(default_factory=list)` --- **`default_timeout_s`** `float | None = 180` --- **`extra_exec_args`** `list[str] = field(default_factory=list)` --- **`fakeroot_for_root`** `bool = True` --- **`timeout_grace_s`** `float = 15` --- ```python nemo_gym.sandbox.providers.apptainer.ApptainerExecConfig.__post_init__() -> None ``` ```python class nemo_gym.sandbox.providers.apptainer.ApptainerProbeConfig( command: str | None = READY_PROBE_COMMAND, expected_stdout: str | None = READY_PROBE_EXPECTED, timeout_s: int = 30, deadline_s: float | None = None, stable_count: int = 1, stable_delay_s: float = 0.0 ) ``` Dataclass Post-create probe settings: a test command confirming the sandbox is usable. **`command`** `str | None = READY_PROBE_COMMAND` --- **`deadline_s`** `float | None = None` --- **`expected_stdout`** `str | None = READY_PROBE_EXPECTED` --- **`stable_count`** `int = 1` --- **`stable_delay_s`** `float = 0.0` --- **`timeout_s`** `int = 30` --- ```python nemo_gym.sandbox.providers.apptainer.ApptainerProbeConfig.__post_init__() -> None ``` ```python class nemo_gym.sandbox.providers.apptainer.ApptainerProvider( exec: nemo_gym.sandbox.providers.apptainer.provider.ApptainerExecConfig | collections.abc.Mapping[str, typing.Any] | None = None, create: nemo_gym.sandbox.providers.apptainer.provider.ApptainerCreateConfig | collections.abc.Mapping[str, typing.Any] | None = None, probe: nemo_gym.sandbox.providers.apptainer.provider.ApptainerProbeConfig | collections.abc.Mapping[str, typing.Any] | None = None, bin_path: str | None = None ) ``` Sandbox provider backed by the local Apptainer CLI. **`_binary`** `= _require_apptainer(bin_path)` --- **`_create_config`** `= _coerce_config(create, ApptainerCreateConfig)` --- **`_exec_config`** `= _coerce_config(exec, ApptainerExecConfig)` --- **`_probe`** `= _coerce_config(probe, ApptainerProbeConfig)` --- **`_semaphore`** `= asyncio.Semaphore(self._exec_config.concurrency)` --- **`_subprocess_env`** `= _apptainer_subprocess_env(bin_path)` --- **`name`** `= 'apptainer'` --- ```python nemo_gym.sandbox.providers.apptainer.ApptainerProvider._cleanup_failed_create_handle( handle: nemo_gym.sandbox.providers.base.SandboxHandle ) -> None ``` async Best-effort teardown of a sandbox that failed verification. ```python nemo_gym.sandbox.providers.apptainer.ApptainerProvider._run( argv: list[str], timeout_s: float | None, stdin: bytes | None = None, daemonize: bool = False ) -> tuple[int, str, str] ``` async Run an apptainer CLI command. Returns (return\_code, stdout, stderr). Enforces timeout via asyncio.wait\_for and kills the whole process group on timeout so child processes do not linger. Bounds concurrency with a shared semaphore. Decodes output with errors="replace". Set `daemonize=True` for commands that fork a long-lived background process (`apptainer instance start`). Such commands hand the started instance a copy of the child's stdout/stderr, so reading those pipes to EOF (`communicate()`) blocks until the *instance* exits — i.e. the call appears to hang until `timeout_s` even though the foreground process finished in under a second. In that mode we capture output to temp files (which the instance may inherit harmlessly) and only wait for the foreground process to exit. ```python nemo_gym.sandbox.providers.apptainer.ApptainerProvider._run_daemonizing( argv: list[str], timeout_s: float | None ) -> tuple[int, str, str] ``` async Run a command that daemonizes a child (e.g. `apptainer instance start`). Captures stdout/stderr to temp files instead of pipes so the long-lived instance inheriting those descriptors cannot wedge the read, then waits only for the foreground process to exit. ```python nemo_gym.sandbox.providers.apptainer.ApptainerProvider._verify_created_handle( handle: nemo_gym.sandbox.providers.base.SandboxHandle ) -> None ``` async Run the readiness probe until the sandbox responds, or raise. * probe.command is None -> skip (no verification). * probe.deadline\_s is None -> single attempt; a failure raises immediately. * probe.deadline\_s is set -> poll until the sandbox passes the probe `stable_count` consecutive times, or the deadline elapses. ```python nemo_gym.sandbox.providers.apptainer.ApptainerProvider.aclose() -> None ``` async No provider-wide resources to close. ```python nemo_gym.sandbox.providers.apptainer.ApptainerProvider.close( handle: nemo_gym.sandbox.providers.base.SandboxHandle ) -> None ``` async Stop the instance and clean up the host staging dir. Runs apptainer instance stop \ If there is no instance --> SUCCESS Removes the host staging dir afterward ```python nemo_gym.sandbox.providers.apptainer.ApptainerProvider.connect( descriptor: collections.abc.Mapping[str, typing.Any] ) -> nemo_gym.sandbox.providers.base.SandboxHandle ``` async ```python nemo_gym.sandbox.providers.apptainer.ApptainerProvider.create( spec: nemo_gym.sandbox.providers.base.SandboxSpec ) -> nemo_gym.sandbox.providers.base.SandboxHandle ``` async Start an apptainer instance and return a ready handle. Steps: 1. Warn once if spec.ttl\_s is set (unsupported by apptainer). 2. Resolve the image source (local .sif path or remote docker://, oras\://, library:// URI) from spec.image. -- just use as is 3. Make a host staging dir (tempfile.mkdtemp), pick mount\_point = self.\_create\_config.mount\_point, generate a unique name = INSTANCE\_NAME\_PREFIX + uuid4().hex. 4. Build argv: \[binary, "instance", "start", \<--bind staging:mount\_point>, \, \, \<--env-file ...>, \_resource\_flags(spec.resources), \, image, name]. 5. await self.\_run(argv, timeout\_s=self.\_create\_config.start\_timeout\_s); on non-zero return, clean up the staging dir and raise ApptainerCreateError(stderr). 6. Build the handle: SandboxHandle(sandbox\_id=name, provider\_name=self.name, raw=\_ApptainerInstance(name, staging\_dir, mount\_point, image)). 7. Verify readiness via self.\_verify\_created\_handle(handle); on failure clean up and raise ApptainerCreateVerificationError. 8. Return the handle. ```python nemo_gym.sandbox.providers.apptainer.ApptainerProvider.download_file( handle: nemo_gym.sandbox.providers.base.SandboxHandle, source_path: str, target_path: pathlib.Path ) -> None ``` async Download one sandbox file to the host. Fast path (source under the bind mount): read directly from the host side of the shared folder. Fallback (arbitrary path): cp inside the container into the shared folder, then read the host side. ```python nemo_gym.sandbox.providers.apptainer.ApptainerProvider.exec( handle: nemo_gym.sandbox.providers.base.SandboxHandle, command: str, cwd: str | None = None, env: dict[str, str] | None = None, timeout_s: int | float | None = None, user: str | int | None = None, stdin: bytes | None = None ) -> nemo_gym.sandbox.providers.base.SandboxExecResult ``` async Run a command inside the instance. Maps the neutral `user` parameter onto apptainer: * None -> run as the default (launching) user. * "root" / 0 -> add --fakeroot (root inside the container). * other user/uid -> --fakeroot + wrap in `su` to switch to that user. `stdin`, when given, is piped to the command's standard input. This is an apptainer-provider extension to the base protocol, useful for feeding large inputs (e.g. prompts) that would exceed the kernel's argv length limit. ```python nemo_gym.sandbox.providers.apptainer.ApptainerProvider.serialize_handle( handle: nemo_gym.sandbox.providers.base.SandboxHandle, scope: str | None = None ) -> dict[str, typing.Any] ``` async Share a local instance with another Gym worker on the same host/UID. This is a trusted control-plane descriptor, not a portable lease. The receiving worker needs the same staging filesystem and Apptainer config. Either handle can stop the instance; callers must coordinate ownership. ```python nemo_gym.sandbox.providers.apptainer.ApptainerProvider.status( handle: nemo_gym.sandbox.providers.base.SandboxHandle ) -> nemo_gym.sandbox.providers.base.SandboxStatus ``` async Return the instance's lifecycle status by querying `apptainer instance list`. Runs apptainer instance list --json On timeout, non-zero, unparseable JSON ---> UNKNOWN Look for the instance name of this sandbox. If it is found --> RUNNING. If it's gone --> STOPPED ```python nemo_gym.sandbox.providers.apptainer.ApptainerProvider.upload_file( handle: nemo_gym.sandbox.providers.base.SandboxHandle, source_path: pathlib.Path, target_path: str ) -> None ``` async Upload one host file into the sandbox. Fast path (target under the bind mount): write directly to the host side of the shared folder. Fallback (arbitrary path): stage into the shared folder, then cp inside the container. ## Docs - [nemo_gym.sandbox.providers.apptainer.provider](https://docs.nvidia.com/nemo/gym/v0.3.0/nemo-gym/nemo_gym/sandbox/providers/apptainer/provider.md)