nemo_gym.sandbox.providers.opensandbox

View as Markdown

OpenSandbox provider package.

Submodules

Package Contents

Classes

NameDescription
OpenSandboxAttributionConfigJob attribution merged into every sandbox’s metadata (Kubernetes labels on the sandbox).
OpenSandboxConnectionConfigOpenSandbox server connection settings.
OpenSandboxCreateConfigOpenSandbox create/reconnect retry settings.
OpenSandboxCreateErrorRaised when OpenSandbox cannot create a sandbox.
OpenSandboxCreateTimeoutErrorRaised when OpenSandbox sandbox creation exceeds the client timeout.
OpenSandboxCreateVerificationErrorRaised when a newly-created sandbox cannot execute a probe command.
OpenSandboxOperationConfigRetry and timeout settings for SDK operations after create.
OpenSandboxProbeConfigPost-create probe settings.
OpenSandboxProviderProvider backed by the OpenSandbox SDK/server API.

API

class nemo_gym.sandbox.providers.opensandbox.OpenSandboxAttributionConfig(
enabled: bool = True,
team: str | None = None,
user: str | None = None,
workload: str | None = None,
run: str | None = None,
key_prefix: str = DEFAULT_ATTRIBUTION_KEY_PREFIX
)
Dataclass

Job attribution merged into every sandbox’s metadata (Kubernetes labels on the sandbox).

OpenSandbox propagates sandbox metadata as Kubernetes labels on the sandbox resources, so attribution is queryable both through the OpenSandbox list API and at the cluster level (e.g. kubectl get pods -l nemo-gym.nvidia.com/team=my-team). key_prefix namespaces the label keys (Kubernetes prefixed-key convention); set it to "" for bare team / user / workload / run keys.

Unset fields are auto-detected: NEMO_GYM_TEAM / NEMO_GYM_USER / NEMO_GYM_WORKLOAD environment variables first, then Slurm job env vars (SLURM_JOB_ACCOUNT / SLURM_JOB_USER / SLURM_JOB_NAME), then the OS login name for user (root is ignored) and the gym CLI’s NEMO_GYM_CONFIG_PATH server instance name for workload. Fields that cannot be resolved are omitted. run scopes sandboxes to one launch of the creating process (NEMO_GYM_RUN_ID, else generated per process and logged) so a run’s sandboxes can be listed and cleaned up exactly. Explicit SandboxSpec.metadata keys always take precedence over attribution keys.

enabled
bool = True
key_prefix
str = DEFAULT_ATTRIBUTION_KEY_PREFIX
run
str | None = None
team
str | None = None
user
str | None = None
workload
str | None = None
nemo_gym.sandbox.providers.opensandbox.OpenSandboxAttributionConfig.__post_init__() -> None
class nemo_gym.sandbox.providers.opensandbox.OpenSandboxConnectionConfig(
domain: str | None = None,
api_key: str | None = None,
protocol: str | None = None,
request_timeout_s: int | None = None,
use_server_proxy: bool = False,
disable_connection_pooling: bool = False,
keepalive_expiry_s: float | None = 3.0,
max_keepalive_connections: int = 20,
max_connections: int | None = 100,
connect_retries: int = 2,
transport_backend: str = 'aiohttp',
tls_verify: bool = False
)
Dataclass

OpenSandbox server connection settings.

With the legacy httpx backend, keepalive_expiry_s must stay below the server’s own keep-alive idle timeout (uvicorn defaults to 5s), or sockets are reused after the server has closed them; null falls back to the SDK’s default transport only when certificate verification is enabled and pooling is not disabled. transport_backend=aiohttp uses Gym’s global client and connector limits; provider-local pooling settings apply only to transport_backend=httpx. tls_verify applies to every connection the provider opens (SDK transport and PTY sockets) and is off by default; set it for endpoints whose certificate the client can verify. domain may carry its scheme (https://sandbox.example). The scheme is moved into protocol and takes precedence over a configured protocol, so every URL the provider builds itself (the PTY WebSocket target) agrees with the SDK’s base URL; the SDK receives the host plus any path prefix (sandbox.example:8080/prefix). The SDK would accept a scheme in domain on its own, but the provider reads protocol directly, hence the normalization here. Only scheme://host[:port][/path-prefix] is accepted: a query string or fragment is a configuration error.

api_key
str | None = None
connect_retries
int = 2
disable_connection_pooling
bool = False
domain
str | None = None
keepalive_expiry_s
float | None = 3.0
max_connections
int | None = 100
max_keepalive_connections
int = 20
protocol
str | None = None
request_timeout_s
int | None = None
tls_verify
bool = False
transport_backend
str = 'aiohttp'
use_server_proxy
bool = False
nemo_gym.sandbox.providers.opensandbox.OpenSandboxConnectionConfig.__post_init__() -> None
class nemo_gym.sandbox.providers.opensandbox.OpenSandboxCreateConfig(
request_timeout_s: int | None = None,
timeout_s: float | None = None,
retries: int = 2,
retry_delay_s: float = 5.0,
retry_max_delay_s: float = 60.0,
image_pull_policy: str | None = DEFAULT_IMAGE_PULL_POLICY,
skip_health_check: bool = False,
connect_attempt_timeout_s: float = 30.0,
connect_poll_s: float = 2.0,
renew_interval_s: float | None = None
)
Dataclass

OpenSandbox create/reconnect retry settings.

connect_attempt_timeout_s
float = 30.0
connect_poll_s
float = 2.0
image_pull_policy
str | None = DEFAULT_IMAGE_PULL_POLICY
renew_interval_s
float | None = None
request_timeout_s
int | None = None
retries
int = 2
retry_delay_s
float = 5.0
retry_max_delay_s
float = 60.0
skip_health_check
bool = False
timeout_s
float | None = None
nemo_gym.sandbox.providers.opensandbox.OpenSandboxCreateConfig.__post_init__() -> None
class nemo_gym.sandbox.providers.opensandbox.OpenSandboxCreateError()

Bases: SandboxCreateError

Raised when OpenSandbox cannot create a sandbox.

class nemo_gym.sandbox.providers.opensandbox.OpenSandboxCreateTimeoutError()

Bases: OpenSandboxCreateError

Raised when OpenSandbox sandbox creation exceeds the client timeout.

class nemo_gym.sandbox.providers.opensandbox.OpenSandboxCreateVerificationError()

Bases: SandboxCreateVerificationError

Raised when a newly-created sandbox cannot execute a probe command.

class nemo_gym.sandbox.providers.opensandbox.OpenSandboxOperationConfig(
retries: int = 3,
retry_delay_s: float = 1.0,
retry_max_delay_s: float = 15.0,
command_retries: int = 0,
close_timeout_s: float | None = 30.0,
pause_resume_timeout_s: float = 600.0,
background_exec: bool = False,
background_poll_initial_s: float = 0.25,
background_poll_interval_s: float = 2.0,
status_poll_timeout_s: float | None = 10.0
)
Dataclass

Retry and timeout settings for SDK operations after create.

background_exec
bool = False
background_poll_initial_s
float = 0.25
background_poll_interval_s
float = 2.0
close_timeout_s
float | None = 30.0
command_retries
int = 0
pause_resume_timeout_s
float = 600.0
retries
int = 3
retry_delay_s
float = 1.0
retry_max_delay_s
float = 15.0
status_poll_timeout_s
float | None = 10.0
nemo_gym.sandbox.providers.opensandbox.OpenSandboxOperationConfig.__post_init__() -> None
class nemo_gym.sandbox.providers.opensandbox.OpenSandboxProbeConfig(
command: str | None = 'printf nemo-gym-sandbox-re...,
expected_stdout: str | None = 'nemo-gym-sandbox-ready',
timeout_s: int = 30,
deadline_s: float | None = None,
stable_count: int = 1,
stable_delay_s: float = 0.0,
user: str | int | None = None
)
Dataclass

Post-create probe settings.

command
str | None = 'printf nemo-gym-sandbox-ready'
deadline_s
float | None = None
expected_stdout
str | None = 'nemo-gym-sandbox-ready'
stable_count
int = 1
stable_delay_s
float = 0.0
timeout_s
int = 30
user
str | int | None = None
nemo_gym.sandbox.providers.opensandbox.OpenSandboxProbeConfig.__post_init__() -> None
class nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider(
connection: nemo_gym.sandbox.providers.opensandbox.provider.OpenSandboxConnectionConfig | collections.abc.Mapping[str, typing.Any] | None = None,
create: nemo_gym.sandbox.providers.opensandbox.provider.OpenSandboxCreateConfig | collections.abc.Mapping[str, typing.Any] | None = None,
probe: nemo_gym.sandbox.providers.opensandbox.provider.OpenSandboxProbeConfig | collections.abc.Mapping[str, typing.Any] | None = None,
operations: nemo_gym.sandbox.providers.opensandbox.provider.OpenSandboxOperationConfig | collections.abc.Mapping[str, typing.Any] | None = None,
attribution: nemo_gym.sandbox.providers.opensandbox.provider.OpenSandboxAttributionConfig | collections.abc.Mapping[str, typing.Any] | None = None,
networking: nemo_gym.sandbox.providers.opensandbox.provider.OpenSandboxNetworkingConfig | collections.abc.Mapping[str, typing.Any] | None = None,
shared_storage: nemo_gym.sandbox.providers.opensandbox.provider.OpenSandboxSharedStorageConfig | collections.abc.Mapping[str, typing.Any] | None = None,
runtime_requirements: nemo_gym.sandbox.providers.opensandbox.provider.OpenSandboxRuntimeRequirementsConfig | collections.abc.Mapping[str, typing.Any] | None = None
)

Provider backed by the OpenSandbox SDK/server API.

_attribution
_connection
_create
= _coerce_config(create, OpenSandboxCreateConfig)
_networking
_operations
_probe
= _coerce_config(probe, OpenSandboxProbeConfig)
_pty_sessions
set[Any] = set()
_renewals
dict[str, Task[None]] = {}
_runtime_requirements
_shared_storage
_transport
Any | None = None
name
= 'opensandbox'
nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._attribution_metadata() -> dict[str, str]
nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._await_sdk_call(
awaitable: typing.Any,
operation: str,
sandbox_id: str,
timeout_s: float | None
) -> typing.Any
async
nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._await_sdk_operation(
operation_factory: typing.Callable[[], typing.Awaitable[typing.Any]],
operation: str,
sandbox_id: str,
timeout_s: float | None,
retries: int | None = None,
is_retryable: typing.Callable[[BaseException], bool] = _is_retryable_sdk_operation...
) -> typing.Any
async
nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._build_transport() -> typing.Any

Use Gym’s global HTTP pool, or the explicitly selected legacy backend.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._cleanup_failed_create_handle(
) -> None
async
nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._command_retry_count() -> int
nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._connect_after_create(
async

Reconnect after SDK create so follow-up calls use a fresh SDK handle.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._connection_config(
request_timeout_s: int | float | None = None
) -> typing.Any
nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._create_once(
async

Create a sandbox through opensandbox.Sandbox.create.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._create_with_retries(
async
nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._exec(
command: str,
cwd: str | None = None,
env: dict[str, str] | None = None,
timeout_s: int | float | None = None,
user: str | int | None = None,
retries: int | None = None
async

Run a command inside an OpenSandbox sandbox.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._exec_background(
command: str,
opts_kwargs: dict[str, typing.Any],
sdk_timeout_s: float | None,
total_timeout_s: int | float | None,
retries: int
async

Run a command as a background execution polled via short requests.

The logs endpoint returns one combined stream, so unlike the foreground path stdout carries both streams and stderr is set only when the sandbox itself reports an error.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._get_transport() -> typing.Any

Return the provider-owned shared transport, building it on first use.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._oom_death_notice(
any_death: bool = False
) -> str | None
async

Briefly poll the sandbox status; describe an OOM kill, else None.

With any_death every terminal state is reported, not just an OOM kill — for callers that need to know whether the sandbox is gone at all, not specifically why. When the backend stops answering it usually takes the control plane a moment to record why, so poll for up to 5s before giving up.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._pty_http_client() -> typing.Any

Return the aiohttp client for one PTY session (same tls_verify as the SDK transport).

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._pty_session_missing(
base_url: str,
headers: dict[str, str],
session_id: str,
request_timeout_s: float | None
) -> bool
async

True only when execd itself reports the PTY session does not exist.

A proxy 404 (route not registered yet) lacks execd’s error code, and a failed check is treated as unknown so the attach proceeds as before.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._pty_target(
) -> tuple[str, dict[str, str], float | None]
async

Resolve the sandbox’s execd base URL, headers and request timeout.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._read_file(
source_path: str
) -> bytes
async

Read one file from an OpenSandbox sandbox.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._resolve_extensions(
extensions: collections.abc.Mapping[str, str]
) -> dict[str, str]

Add the configured default image pull policy to SDK create extensions.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._retire_closed_pty_sessions() -> None
async

Release sessions that ended on their own; their aiohttp client is only freed by close(). Called from create/attach so the tracking set cannot grow without bound.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._start_renewal(
ttl_s: int | float
) -> None
nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._stop_renewal(
sandbox_id: str
) -> BaseException | None
async
nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._submit_command(
operation_factory: typing.Callable[[], typing.Awaitable[typing.Any]],
operation: str,
sandbox_id: str,
timeout_s: float | None,
retries: int
) -> typing.Any
async

Retry backend-connect 502s that command_retries deliberately skips.

A proxy 502 is a TCP-connect failure: the command never reached execd, so retrying under operations.retries cannot double-run it (unlike a real command failure). When that budget is exhausted the backend is dead, so raise a typed error and fail fast instead of retrying for hours.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._verify_created_handle(
) -> None
async
nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider._write_file(
target_path: str,
data: str | bytes
) -> None
async

Write one file into an OpenSandbox sandbox.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.aclose() -> None
async

Close provider-owned resources.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.attach_pty(
session_id: str,
takeover: bool = True,
since: int | None = None
async

Re-attach to an existing execd PTY session by id.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.close(
) -> None
async

Terminate the sandbox and close local SDK resources.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.configure_runtime(
cap_add: tuple[str, ...],
shm_size: int | None
) -> None
async

Probe capabilities and verify the shared memory allocated at creation.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.connect(
descriptor: collections.abc.Mapping[str, typing.Any]
async

Rebuild a live handle from an OpenSandbox sandbox id via the SDK.

Running sandboxes are health-checked unless the caller opts out. A paused sandbox has no exec daemon to check; resume rebuilds its endpoints and performs the health check instead.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.create(
async

Create one sandbox through the configured OpenSandbox path.

Job attribution keys (team / user / workload / run) are merged into the spec’s metadata (explicit spec keys win) so every sandbox is attributable via its labels.

async

Open an interactive execd PTY session inside a sandbox.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.download_file(
source_path: str,
target_path: pathlib.Path
) -> None
async

Download one file from an OpenSandbox sandbox.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.endpoint(
port: int
async

Resolve one client-reachable direct or server-proxied service URL.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.exec(
command: str,
cwd: str | None = None,
env: dict[str, str] | None = None,
timeout_s: int | float | None = None,
user: str | int | None = None
async

Run a command inside an OpenSandbox sandbox.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.exec_with_background_services(
command: str,
cwd: str | None = None,
timeout_s: int | float | None = None
async

Preserve background services that redirect their stdout and stderr.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.forward_ports(
target_address: str,
ports: tuple[int, ...],
ready_file: str
) -> None
async

Run loopback listeners in a foreground command owned by the collection.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.network_address(
) -> str
async

Resolve a direct container IP, independently of client proxy mode.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.pause(
) -> None
async

Pause a sandbox and wait until it reports paused.

Local PTY clients are detached first, while execd can still answer the close handshake; server sessions are never deleted, so they remain attachable if the pause request fails. After resume, the Kubernetes backend has replaced the runtime (open a new PTY); the Docker backend thawed it (re-attach by id).

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.resume(
) -> None
async

Resume a paused sandbox and rebuild its SDK clients and endpoints.

One pause_resume_timeout_s deadline covers the request, endpoint rebuild and readiness check. On timeout the server-side state is unknown: reconnect and check status() before retrying. See pause() for what happens to PTY sessions.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.serialize_handle(
scope: str | None = None
) -> dict[str, typing.Any]
async

Return a descriptor for reattaching to this sandbox by id.

OpenSandbox sandboxes are reachable by id from any process that has the connection config, so the id alone is enough to reconnect and no sandbox server is needed to share one. scope is ignored: OpenSandbox has no lease concept of its own.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.set_hosts(
hosts: collections.abc.Mapping[str, str]
) -> None
async

Append validated peer aliases to the sandbox hosts file.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.shared_volume_metadata() -> dict[str, str]

Return placement metadata required by the configured shared storage.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.shared_volume_options(
source: str | None,
target: str,
read_only: bool = False
) -> dict[str, typing.Any]

Mount a project directory, or the shared root for bootstrap when source is None.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.status(
async

Return the current OpenSandbox lifecycle status.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.upload_file(
source_path: pathlib.Path,
target_path: str
) -> None
async

Upload one local file into an OpenSandbox sandbox.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.validate_networking() -> None

Require explicit deployment support for inter-sandbox networking.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.validate_port_forwarding() -> None

Require explicit deployment support for loopback TCP listeners.

nemo_gym.sandbox.providers.opensandbox.OpenSandboxProvider.validate_runtime_requirements(
cap_add: tuple[str, ...],
shm_size: int | None
) -> dict[str, str]

Reject requirements without an operator-configured implementation.