nemo_gym.sandbox.providers.opensandbox
nemo_gym.sandbox.providers.opensandbox
OpenSandbox provider package.
Submodules
nemo_gym.sandbox.providers.opensandbox.cleanup_sandboxesnemo_gym.sandbox.providers.opensandbox.failuresnemo_gym.sandbox.providers.opensandbox.providernemo_gym.sandbox.providers.opensandbox.ptynemo_gym.sandbox.providers.opensandbox.resource_telemetrynemo_gym.sandbox.providers.opensandbox.snapshots
Package Contents
Classes
Data
API
OpenSandbox server connection settings.
With the legacy httpx backend, keepalive_expiry_s must stay below the
server’s own keep-alive idle timeout (uvicorn defaults to 5s), or sockets are reused after the
server has closed them; null falls back to the SDK’s default transport only
when certificate verification is enabled and pooling is not disabled.
transport_backend=aiohttp uses Gym’s global client and connector limits;
provider-local pooling settings apply only to transport_backend=httpx.
With the explicit legacy combination transport_backend=httpx,
keepalive_expiry_s=None and tls_verify=True (pooling enabled), the
SDK owns its default transport. SDK 0.1.15 then discards command-status 410
bodies; lifecycle evidence remains available independently of that response.
tls_verify applies to every connection the provider opens (SDK transport
and PTY sockets) and is off by
default; set it for endpoints whose certificate the client can verify.
domain may carry its scheme (https://sandbox.example). The scheme is
moved into protocol and takes precedence over a configured protocol,
so every URL the provider builds itself (the PTY WebSocket target) agrees
with the SDK’s base URL; the SDK receives the host plus any path prefix
(sandbox.example:8080/prefix). The SDK would accept a scheme in
domain on its own, but the provider reads protocol directly, hence
the normalization here. Only scheme://host[:port][/path-prefix] is
accepted: a query string or fragment is a configuration error.
OpenSandbox create/reconnect retry settings.
Retry and timeout settings for SDK operations after create.
Post-create probe settings.
Provider backed by the OpenSandbox SDK/server API.
Use Gym’s global HTTP pool, or the explicitly selected legacy backend.
Reconnect after SDK create so follow-up calls use a fresh SDK handle.
Opt-in OpenSandbox 1.1.0 text fallback; no supported structured equivalent is assumed.
The per-attempt budget and the SDK request timeout for one create of spec.
The create request blocks until the pod is allocated, so a configured request timeout
(create.request_timeout_s, else connection.request_timeout_s) is capped at the spec’s ready
timeout: an unschedulable pod then fails within one ready timeout instead of the full request
timeout. When none is configured the SDK’s own default stays, as the connection config the create
is given lives on the handle for the sandbox’s life.
The SDK’s readiness wait starts only once the request returns, so the attempt budget must hold the
request bound plus ready_timeout_s. A configured create.timeout_s is respected as is (a
warning names it when it is below that sum); otherwise the budget is that sum plus a slack.
Create a sandbox through opensandbox.Sandbox.create.
Run a command inside an OpenSandbox sandbox.
Register this command’s poll for checkpoint polling scopes, and release it on every exit.
Run a command as a background execution polled via short requests.
The logs endpoint returns one combined stream, so unlike the foreground
path stdout carries both streams and stderr is set only when the
sandbox itself reports an error.
Gym enforces total_timeout_s itself: once its own deadline passes
while the command is still running, it interrupts the command and
returns the timeout result, as the session path does. execd reports its
own timeout kill only as a signal death, so its timer is moved past
Gym’s deadline and serves as a backstop.
One setup.commands entry as root, or as the image’s own user when the sandbox cannot switch
identity.
An image with a non-root USER on a cell that drops CAP_SETUID refuses the switch before the command
runs (nemo_gym.sandbox.transfer.exec_as_root degrades the same way), so a command written to no-op where
it does not apply gets to no-op, and one that needs root there fails on its own result, as before.
Return the provider-owned shared transport, building it on first use.
Interrupt a background command still running at Gym’s deadline; whether the interrupt was accepted.
Apply the same idempotent deletion and permission guards to handles and orphan cleanup.
One request to the server’s management API over Gym’s shared aiohttp client, which the SDK handle does
not expose, within operations.snapshot_request_timeout_s.
Returns the HTTP status, the JSON object body ({} for an error status or an empty body) and, for an
error status, the start of the body’s text ("" otherwise). The request is sent once: a transport
failure, a reply cut off mid-body included, is raised for the caller’s own retries.
Return the aiohttp client for one PTY session (same tls_verify as the SDK transport).
True only when execd itself reports the PTY session does not exist.
A proxy 404 (route not registered yet) lacks execd’s error code, and a failed check is treated as unknown so the attach proceeds as before.
Resolve the sandbox’s execd base URL, headers and request timeout.
Reap what a timed-out create left behind and raise the error that describes it.
Report provider_create_pending only for an observed Pending sandbox; otherwise report
provider_create_timeout. The create policy chooses retry, with fail_fast_on_pending
preventing another attempt for a Pending sandbox when configured.
request is the create request if still open; it is cancelled once its sandbox is deleted.
Find and delete the sandbox a timed-out create request left behind; return what was found.
The request never returned, so the client holds no sandbox id: the sandbox is found through the
run’s attribution metadata and this attempt’s unique key. Each match is deleted and described
(id, state, reason, message) so the caller can tell a pod that never left
PENDING from one that was merely slow. Failures are logged, never raised: the create’s own
timeout error is the one the caller must see.
Add the configured default image pull policy to SDK create extensions.
Release sessions that ended on their own; their aiohttp client is
only freed by close(). Called from create/attach so the tracking
set cannot grow without bound.
One evidence path for exec and PTY, bounded by the caller’s existing deadline.
Lifecycle reasons do not identify the affected container, so even OOMKilled establishes only unavailability. A recognized control-plane container report or optional inspect can confirm OOM for the current user container.
Renew the sandbox’s TTL every renew_interval_s (jittered by up to 10% earlier) until cancelled.
The sandbox’s TTL has been running since created_at (the event loop’s time when the SDK create
request was sent, which is when the server starts it), so the first renewal comes that much sooner, and at
once if more than one interval has passed.
Without a lease every renewal sets the TTL to ttl_s again, and one that fails after its retries, or
that finds the sandbox gone, ends the loop and fails the sandbox’s later operations. With a lease
(renew_ttl_s) every renewal sets it to the lease, never past lifetime_s (the spec’s ttl_s) from
creation; a renewal that fails is logged and tried again within what is left of the TTL the service last
accepted (see _renewal_retry_delay), and one that finds the sandbox gone (stopped or deleted elsewhere)
ends the loop quietly.
Submit once unless the request never left the client.
Neither a 502 nor a timeout proves non-delivery, and command retry settings cannot
override that: the provider has no idempotency key or authoritative non-delivery
receipt. A connection that never opened is that receipt, so such a failure, and only
such a failure, is sent again within operations.retries.
Wait for the next poll or a scope change, including a wakeup delivered before this call.
Hold a command’s poll while a parked scope is open on its sandbox, but not past the command’s
deadline (the event loop’s time): a park does not suspend the command’s own time limit.
Write one file into an OpenSandbox sandbox.
Close provider-owned resources.
Re-attach to an existing execd PTY session by id.
Wake the background command polls of this sandbox and poll at least every
operations.checkpoint_background_poll_interval_s until the scope exits.
For a harness being stopped or paused for a checkpoint, so its command is seen to end promptly.
Terminate the sandbox and close local SDK resources.
Probe capabilities and verify the shared memory allocated at creation.
Rebuild a live handle from an OpenSandbox sandbox id via the SDK.
Running sandboxes are health-checked unless the caller opts out. A paused sandbox has no exec daemon to check; resume rebuilds its endpoints and performs the health check instead.
Create one sandbox through the configured OpenSandbox path.
Job attribution keys (team / user / workload / run) are merged into the
spec’s metadata (explicit spec keys win) so every sandbox is attributable via its labels.
Create a sandbox from a snapshot: OpenSandbox takes snapshot_id in place of the image.
Raises SandboxSnapshotNotFoundError when the server answers that it has no such snapshot: with its
SNAPSHOT::NOT_FOUND code, or with a bare HTTP 404, which is what the service sends for a create from a
snapshot it has deleted. A 404 with any other error code keeps its own type: it is about something else,
such as the new sandbox being gone by the time its endpoint is looked up. A snapshot whose delete was only
just accepted can still create a sandbox for a short while.
Open an interactive execd PTY session inside a sandbox.
Delete a snapshot over the management API; one the API confirms is already gone is not an error.
A 404 to the delete is confirmed with a lookup of the same snapshot: only a 404 to both means it is gone.
A lookup that still finds the snapshot makes the 404 a transient answer. Transient answers (429, 5xx,
connection errors and that 404, for the delete or its confirming lookup) are retried
operations.retries times with the operation backoff, one request per try. A snapshot still there after
them, or any other answer, raises SandboxSnapshotError; a 409 to the delete, which the API answers
while something still uses the snapshot, raises SandboxSnapshotInUseError at once. The whole delete
has operations.snapshot_delete_timeout_s; one that is not done by then raises SandboxSnapshotError.
Download one file from an OpenSandbox sandbox, streamed to disk in chunks.
Resolve one client-reachable direct or server-proxied service URL.
Run a command inside an OpenSandbox sandbox.
Preserve background services that redirect their stdout and stderr.
Run loopback listeners in a foreground command owned by the collection.
Every snapshot whose name starts with prefix.
The management API’s name query parameter matches a whole name, not a prefix, so this reads every page
of the listing, other runs’ snapshots included, and filters here. Pages are collected before the caller
deletes anything, so its own deletes do not shift them. The listing is paged by offset, though, so a
snapshot another run creates or deletes meanwhile can move one across a page boundary and out of this
listing; a caller that sweeps leftovers finds it in its next listing. A page request is retried
operations.retries times with the operation backoff for 429, 5xx, connection errors and a reply cut off
mid-body, one request per try; a page that still fails, any other answer, or a malformed listing raises
SandboxSnapshotError instead of returning a partial listing. So does a listing that is not complete
within operations.snapshot_list_timeout_s. A page without items ends the listing whatever it says of
a next page; a page with items must say whether another follows.
Resolve a direct container IP, independently of client proxy mode.
Hold the background command polls of this sandbox, without asking it, until the scope exits.
For a harness launch paused for a checkpoint: the sandbox is being snapshotted and may not answer for a
while. A status request that fails after the scope opened is asked again once it exits, with a fresh retry
allowance. A park does not extend a command’s time limit: a command whose client wait (its timeout plus
operations.command_timeout_headroom_s) runs out while parked fails then, with its outcome unknown. Do
not wait inside the scope for a command of this sandbox: its polls are held too.
Pause a sandbox and wait until it reports paused.
Local PTY clients are detached first, while execd can still answer the close handshake; server sessions are never deleted, so they remain attachable if the pause request fails. After resume, the Kubernetes backend has replaced the runtime (open a new PTY); the Docker backend thawed it (re-attach by id).
Compose’s pull_policy as the per-sandbox image pull policy extension (_resolve_extensions lets a
per-sandbox value win over the deployment default).
Read effective requests and limits from the deployment’s inspection endpoint.
Read execd’s existing snapshot; preserve its timestamp and measurement names.
Resume a paused sandbox and rebuild its SDK clients and endpoints.
One pause_resume_timeout_s deadline covers the request, endpoint
rebuild and readiness check. On timeout the server-side state is
unknown: reconnect and check status() before retrying. See
pause() for what happens to PTY sessions.
Return a descriptor for reattaching to this sandbox by id.
OpenSandbox sandboxes are reachable by id from any process that has the
connection config, so the id alone is enough to reconnect and no sandbox
server is needed to share one. scope is ignored: OpenSandbox has no
lease concept of its own.
Append validated peer aliases to the sandbox hosts file.
Return placement metadata required by the configured shared storage.
Mount a project directory, or the shared root for bootstrap when source is None.
Start a snapshot of a running sandbox and return its id at once.
Creation is asynchronous on the server: the snapshot can create sandboxes only once
wait_snapshot_ready returns. The sandbox keeps running either way. Not retried: a repeated request
could leave a second snapshot nobody knows the id of.
Return the current OpenSandbox lifecycle status.
Upload one local file into an OpenSandbox sandbox.
A foreground command holds one of the shared HTTP pool’s connections for as long as it runs, so
max_connections running commands leave an upload none; a background command is polled and holds none.
Require explicit deployment support for inter-sandbox networking.
Require explicit deployment support for loopback TCP listeners.
Reject requirements without an operator-configured implementation.
Poll the management API until the snapshot is Ready, within operations.snapshot_timeout_s.
A snapshot that fails, reports no state, or is not ready in time raises SandboxSnapshotFailedError.
Transient answers are waited out within the timeout: 429, 5xx, connection errors and a reply cut off
mid-body at any time, and a 404 before the first successful lookup for
operations.snapshot_lookup_grace_s (the API can answer 404 for a snapshot it has only just created).
A 404 past that grace, or once the snapshot was found, raises SandboxSnapshotNotFoundError at once, and
any other answer SandboxSnapshotError.