nemo_gym.sandbox.providers.openshell.provider

OpenShell sandbox provider: sandboxes managed by an OpenShell gateway (github.com/NVIDIA/OpenShell).

The provider talks to the gateway’s gRPC control plane through the synchronous openshell SDK; blocking SDK calls run on a thread pool bounded by exec.concurrency. The client and thread pool are cached at module scope keyed on the connection config, so concurrent sandboxes created from identical provider configs share one gRPC channel and one pool instead of allocating one per sandbox. The SDK has no file-transfer API, so uploads stream bytes through exec stdin (chunked to stay under the gateway’s gRPC message size limit) and downloads round-trip through base64 on the sandbox’s stdout.

Module Contents

Classes

Name	Description
`OpenShellConnectionConfig`	Gateway connection settings. Defaults target a local plaintext gateway (deploy/docker compose).
`OpenShellCreateConfig`	-
`OpenShellCreateError`	Raised when the OpenShell gateway cannot create a sandbox.
`OpenShellCreateVerificationError`	Raised when a new sandbox fails its readiness probe.
`OpenShellExecConfig`	-
`OpenShellOperationsConfig`	-
`OpenShellProbeConfig`	-
`OpenShellProvider`	Sandbox provider backed by an OpenShell gateway’s gRPC control plane.
`OpenShellProviderOptions`	Validated per-sandbox options carried in `SandboxSpec.provider_options`.
`_OpenShellSandbox`	-
`_SharedClientState`	One gRPC client + worker pool shared by every provider with the same connection config.

Functions

Name	Description
`_acquire_shared_client`	-
`_build_client`	-
`_coerce_config`	-
`_delete_and_wait`	-
`_delete_sandbox`	Delete one sandbox and (optionally) wait until it is gone, logging the outcome; runs in a worker thread.
`_delete_state`	-
`_exec_timed_out`	Whether the exec stream ended at the `timeout_seconds` the provider passed it.
`_forget_delete`	-
`_format_conditions`	The sandbox’s reported conditions as a message suffix (the gateway’s own reason, e.g. a pull failure).
`_generate_sandbox_name`	Return a DNS-safe name within OpenShell’s routable-name limit.
`_grpc_status_code`	The `grpc.StatusCode` of an RPC error, else None.
`_is_already_exists`	-
`_is_delete_retryable`	Transient transport failures, or FAILED_PRECONDITION while a create’s driver operation is pending.
`_is_grpc_error`	-
`_is_grpc_timeout`	-
`_is_not_found`	-
`_is_retryable_create_error`	Whether a CreateSandbox RPC failure is likely transient (safe to retry with the same name).
`_is_runtime_failure`	-
`_is_sdk_error`	-
`_normalize_image`	-
`_phase`	-
`_phase_to_status_map`	SandboxPhase -> SandboxStatus, built from the SDK’s generated proto constants.
`_release_shared_client`	-
`_require_openshell`	-
`_wait_deleted`	Poll GetSandbox until NOT_FOUND (transient RPC failures keep polling until the deadline).

Data

DEFAULT_UPLOAD_CHUNK_BYTES

EXEC_TIMEOUT_SLACK_FRACTION

EXEC_TIMEOUT_SLACK_S

LOGGER

MAX_DELETE_ATTEMPTS

MAX_SANDBOX_NAME_LENGTH

READY_PROBE_COMMAND

READY_PROBE_EXPECTED

SANDBOX_LABEL

SANDBOX_NAME_PREFIX

SANDBOX_NAME_RANDOM_HEX_LENGTH

SANDBOX_RUNTIME_RETURN_CODE

_SHARED_CLIENTS_LOCK

API

class nemo_gym.sandbox.providers.openshell.OpenShellConnectionConfig(
    endpoint: str = 'localhost:8080',
    workspace: str = 'default',
    bearer_token: str | None = None,
    tls_ca_path: str | None = None,
    tls_cert_path: str | None = None,
    tls_key_path: str | None = None,
    request_timeout_s: float = 30.0
)

Dataclass

Gateway connection settings. Defaults target a local plaintext gateway (deploy/docker compose).

bearer_token

str | None = None

endpoint

str = 'localhost:8080'

request_timeout_s

float = 30.0

tls_ca_path

str | None = None

tls_cert_path

str | None = None

tls_key_path

str | None = None

workspace

str = 'default'

nemo_gym.sandbox.providers.openshell.OpenShellConnectionConfig.__post_init__() -> None

class nemo_gym.sandbox.providers.openshell.OpenShellCreateConfig(
    ready_timeout_s: float = 300,
    poll_interval_s: float = 1.0,
    retries: int = 2,
    retry_delay_s: float = 1.0,
    retry_max_delay_s: float = 30.0
)

Dataclass

poll_interval_s

float = 1.0

ready_timeout_s

float = 300

retries

int = 2

retry_delay_s

float = 1.0

retry_max_delay_s

float = 30.0

nemo_gym.sandbox.providers.openshell.OpenShellCreateConfig.__post_init__() -> None

class nemo_gym.sandbox.providers.openshell.OpenShellCreateError()

Bases: SandboxCreateError

Raised when the OpenShell gateway cannot create a sandbox.

class nemo_gym.sandbox.providers.openshell.OpenShellCreateVerificationError()

Bases: SandboxCreateVerificationError

Raised when a new sandbox fails its readiness probe.

class nemo_gym.sandbox.providers.openshell.OpenShellExecConfig(
    default_timeout_s: float | None = 180,
    concurrency: int = 32,
    exec_shell: str = '/bin/sh',
    upload_chunk_bytes: int = DEFAULT_UPLOAD_CHUNK_BYTES
)

Dataclass

concurrency

int = 32

default_timeout_s

float | None = 180

exec_shell

str = '/bin/sh'

upload_chunk_bytes

int = DEFAULT_UPLOAD_CHUNK_BYTES

nemo_gym.sandbox.providers.openshell.OpenShellExecConfig.__post_init__() -> None

class nemo_gym.sandbox.providers.openshell.OpenShellOperationsConfig(
    delete_rpc_timeout_s: float = 120,
    delete_concurrency: int = 32,
    close_wait_deleted: bool = True,
    close_timeout_s: float = 60,
    poll_interval_s: float = 1.0
)

Dataclass

close_timeout_s

float = 60

close_wait_deleted

bool = True

delete_concurrency

int = 32

delete_rpc_timeout_s

float = 120

poll_interval_s

float = 1.0

nemo_gym.sandbox.providers.openshell.OpenShellOperationsConfig.__post_init__() -> None

class nemo_gym.sandbox.providers.openshell.OpenShellProbeConfig(
    command: str | None = READY_PROBE_COMMAND,
    expected_stdout: str | None = READY_PROBE_EXPECTED,
    timeout_s: int = 30,
    deadline_s: float | None = 60,
    stable_count: int = 1,
    stable_delay_s: float = 1.0
)

Dataclass

command

str | None = READY_PROBE_COMMAND

deadline_s

float | None = 60

expected_stdout

str | None = READY_PROBE_EXPECTED

stable_count

int = 1

stable_delay_s

float = 1.0

timeout_s

int = 30

nemo_gym.sandbox.providers.openshell.OpenShellProbeConfig.__post_init__() -> None

class nemo_gym.sandbox.providers.openshell.OpenShellProvider(
    connection: nemo_gym.sandbox.providers.openshell.provider.OpenShellConnectionConfig | collections.abc.Mapping[str, typing.Any] | None = None,
    create: nemo_gym.sandbox.providers.openshell.provider.OpenShellCreateConfig | collections.abc.Mapping[str, typing.Any] | None = None,
    exec: nemo_gym.sandbox.providers.openshell.provider.OpenShellExecConfig | collections.abc.Mapping[str, typing.Any] | None = None,
    probe: nemo_gym.sandbox.providers.openshell.provider.OpenShellProbeConfig | collections.abc.Mapping[str, typing.Any] | None = None,
    operations: nemo_gym.sandbox.providers.openshell.provider.OpenShellOperationsConfig | collections.abc.Mapping[str, typing.Any] | None = None,
    attribution: nemo_gym.sandbox.attribution.AttributionConfig | collections.abc.Mapping[str, typing.Any] | None = None
)

Sandbox provider backed by an OpenShell gateway’s gRPC control plane.

_attribution

= _coerce_config(attribution, AttributionConfig)

_connection

_create_config

= _coerce_config(create, OpenShellCreateConfig)

_delete_shared

_exec_config

= _coerce_config(exec, OpenShellExecConfig)

_operations

_probe

= _coerce_config(probe, OpenShellProbeConfig)

_shared

name

= 'openshell'

nemo_gym.sandbox.providers.openshell.OpenShellProvider._build_policy(
    policy: str | collections.abc.Mapping[str, typing.Any]
) -> typing.Any

nemo_gym.sandbox.providers.openshell.OpenShellProvider._build_sandbox_spec(
    spec: nemo_gym.sandbox.providers.base.SandboxSpec,
    image: str | None,
    options: nemo_gym.sandbox.providers.openshell.provider.OpenShellProviderOptions
) -> typing.Any

nemo_gym.sandbox.providers.openshell.OpenShellProvider._call(
    func: typing.Any,
    args: typing.Any = (),
    kwargs: typing.Any = {}
) -> typing.Any

async

nemo_gym.sandbox.providers.openshell.OpenShellProvider._call_create(
    name: str,
    workspace: str,
    kwargs: typing.Any = {}
) -> typing.Any

async

Run CreateSandbox on a worker. Cancelling the await does not stop a call already running, so a create that commits after the cancel is deleted when it returns.

nemo_gym.sandbox.providers.openshell.OpenShellProvider._cleanup_failed_create_handle(
    handle: nemo_gym.sandbox.providers.base.SandboxHandle
) -> None

async

nemo_gym.sandbox.providers.openshell.OpenShellProvider._create_sandbox_with_retries(
    pb_spec: typing.Any,
    name: str,
    labels: dict[str, str]
) -> typing.Any

async

Issue CreateSandbox, retrying transient gRPC failures with the same name.

Retrying with the same name is safe: if an earlier attempt actually committed, the retry fails ALREADY_EXISTS and the sandbox is recovered via GetSandbox.

nemo_gym.sandbox.providers.openshell.OpenShellProvider._reap_orphans() -> None

async

Delete this process’s orphaned sandboxes: ones it no longer tracks, for example a create cancelled after the gateway committed it. Selected by the process label, which needs attribution to be enabled.

nemo_gym.sandbox.providers.openshell.OpenShellProvider._submit_delete(
    inst: nemo_gym.sandbox.providers.openshell.provider._OpenShellSandbox
) -> None

nemo_gym.sandbox.providers.openshell.OpenShellProvider._timed_exec(
    args: typing.Any = (),
    kwargs: typing.Any = {}
) -> tuple[typing.Any, float]

Run the SDK exec and time it inside the worker thread, so executor queueing is not counted.

nemo_gym.sandbox.providers.openshell.OpenShellProvider._verify_created_handle(
    handle: nemo_gym.sandbox.providers.base.SandboxHandle
) -> None

async

Poll the readiness probe until it passes stable_count times or the deadline elapses.

nemo_gym.sandbox.providers.openshell.OpenShellProvider._wait_ready(
    handle: nemo_gym.sandbox.providers.base.SandboxHandle,
    timeout_s: int | float
) -> None

async

Poll GetSandbox until READY, raising on the ERROR/DELETING phases or the deadline.

nemo_gym.sandbox.providers.openshell.OpenShellProvider.aclose() -> None

async

Release the shared client/pool (closed for real when the last provider releases it).

Does not wait for deletes: they run on their own workers. When no sandbox of this process is still live or being created, it first deletes this process’s orphaned sandboxes (see _reap_orphans).

nemo_gym.sandbox.providers.openshell.OpenShellProvider.close(
    handle: nemo_gym.sandbox.providers.base.SandboxHandle
) -> None

async

Hand the sandbox’s delete to a worker thread and return without waiting for the gateway.

The worker deletes, waits until the sandbox is gone when operations.close_wait_deleted is set, and retries transient failures. It is already queued when this returns and runs to completion at interpreter exit even if the event loop is gone. Every outcome is logged with the sandbox name; a delete that fails for good is logged as possibly leaked and then forgotten.

nemo_gym.sandbox.providers.openshell.OpenShellProvider.create(
    spec: nemo_gym.sandbox.providers.base.SandboxSpec
) -> nemo_gym.sandbox.providers.base.SandboxHandle

async

Create a sandbox through the gateway, wait for the READY phase, then probe exec readiness.

spec.image is optional (the gateway’s configured default image is used when unset). spec.ttl_s is not enforced (OpenShell sandboxes live until deleted) and only logs a warning. spec.entrypoint is unsupported: the OpenShell supervisor owns the sandbox entrypoint. spec.provider_options accepts providers (OpenShell credential-provider names), policy (a SandboxPolicy mapping or YAML path), and template_resources / driver_config (free-form driver passthrough Structs). A half-created sandbox is deleted on any failure.

nemo_gym.sandbox.providers.openshell.OpenShellProvider.download_file(
    handle: nemo_gym.sandbox.providers.base.SandboxHandle,
    source_path: str,
    target_path: pathlib.Path
) -> None

async

Download one sandbox file via a base64 round-trip (binary-safe over the text exec stream).

The whole file is buffered in memory (inflated 4/3 by base64), so this is intended for small-to-medium artifacts rather than large archives.

nemo_gym.sandbox.providers.openshell.OpenShellProvider.exec(
    handle: nemo_gym.sandbox.providers.base.SandboxHandle,
    command: str,
    cwd: str | None = None,
    env: dict[str, str] | None = None,
    timeout_s: int | float | None = None,
    user: str | int | None = None,
    stdin: bytes | None = None
) -> nemo_gym.sandbox.providers.base.SandboxExecResult

async

Run <shell> -c <command> through the gateway’s streaming exec; never raises for command failure.

The timeout is enforced by the gateway (timeout_seconds); the SDK extends its gRPC deadline past it. user is ignored with a warning: the OpenShell exec API has no user field, so commands run as the sandbox’s default user.

nemo_gym.sandbox.providers.openshell.OpenShellProvider.status(
    handle: nemo_gym.sandbox.providers.base.SandboxHandle
) -> nemo_gym.sandbox.providers.base.SandboxStatus

async

Sandbox phase via GetSandbox (missing -> STOPPED; RPC failure -> UNKNOWN).

nemo_gym.sandbox.providers.openshell.OpenShellProvider.upload_file(
    handle: nemo_gym.sandbox.providers.base.SandboxHandle,
    source_path: pathlib.Path,
    target_path: str
) -> None

async

Upload one local file by streaming its bytes through exec stdin (creates the parent dir).

Bytes are sent in exec.upload_chunk_bytes chunks because each chunk travels as a single gRPC message that must stay under the gateway’s max decode size.

class nemo_gym.sandbox.providers.openshell.OpenShellProviderOptions(
    providers: list[str] = list(),
    policy: typing.Any | None = None,
    template_resources: dict[str, typing.Any] = dict(),
    driver_config: dict[str, typing.Any] = dict()
)

Dataclass

Validated per-sandbox options carried in SandboxSpec.provider_options.

driver_config

dict[str, Any] = field(default_factory=dict)

policy

Any | None = None

providers

list[str] = field(default_factory=list)

template_resources

dict[str, Any] = field(default_factory=dict)

nemo_gym.sandbox.providers.openshell.OpenShellProviderOptions.from_mapping(
    options: collections.abc.Mapping[str, typing.Any]
) -> nemo_gym.sandbox.providers.openshell.provider.OpenShellProviderOptions

classmethod

class nemo_gym.sandbox.providers.openshell.provider._OpenShellSandbox(
    name: str,
    sandbox_id: str,
    workspace: str,
    image: str | None,
    env: dict[str, str] = dict(),
    workdir: str | None = None
)

Dataclass

env

dict[str, str] = field(default_factory=dict)

image

str | None

name

str

sandbox_id

str

workdir

str | None = None

workspace

str

class nemo_gym.sandbox.providers.openshell.provider._SharedClientState(
    key: tuple[typing.Any, ...],
    client: typing.Any,
    executor: concurrent.futures.ThreadPoolExecutor,
    refcount: int = 0
)

Dataclass

One gRPC client + worker pool shared by every provider with the same connection config.

executor

ThreadPoolExecutor

key

tuple[Any, ...]

refcount

int = 0

nemo_gym.sandbox.providers.openshell.provider._acquire_shared_client(
    connection: nemo_gym.sandbox.providers.openshell.provider.OpenShellConnectionConfig,
    concurrency: int
) -> nemo_gym.sandbox.providers.openshell.provider._SharedClientState

nemo_gym.sandbox.providers.openshell.provider._build_client(
    connection: nemo_gym.sandbox.providers.openshell.provider.OpenShellConnectionConfig
) -> typing.Any

nemo_gym.sandbox.providers.openshell.provider._coerce_config(
    value: typing.Any,
    config_cls: type[typing.Any]
) -> typing.Any

nemo_gym.sandbox.providers.openshell.provider._delete_and_wait(
    client: typing.Any,
    inst: nemo_gym.sandbox.providers.openshell.provider._OpenShellSandbox,
    operations: nemo_gym.sandbox.providers.openshell.provider.OpenShellOperationsConfig
) -> None

nemo_gym.sandbox.providers.openshell.provider._delete_sandbox(
    client: typing.Any,
    inst: nemo_gym.sandbox.providers.openshell.provider._OpenShellSandbox,
    operations: nemo_gym.sandbox.providers.openshell.provider.OpenShellOperationsConfig
) -> None

Delete one sandbox and (optionally) wait until it is gone, logging the outcome; runs in a worker thread.

A client deadline does not cancel a delete the gateway has already started, so after one the worker waits for the sandbox to disappear and only re-issues the delete if it is still there; allow_missing makes the re-issue idempotent. Transient transport failures and FAILED_PRECONDITION (a create’s driver operation is still pending) are retried until operations.close_timeout_s. Each job can use up to MAX_DELETE_ATTEMPTS delete deadlines, plus retry delays and the deletion wait.

nemo_gym.sandbox.providers.openshell.provider._delete_state(
    connection: nemo_gym.sandbox.providers.openshell.provider.OpenShellConnectionConfig,
    concurrency: int
) -> nemo_gym.sandbox.providers.openshell.provider._SharedClientState

nemo_gym.sandbox.providers.openshell.provider._exec_timed_out(
    return_code: int,
    elapsed_s: float | None,
    timeout_s: int | None
) -> bool

Whether the exec stream ended at the timeout_seconds the provider passed it.

The 0.1.x exec stream reports the timeout as exit code 124 with no timeout flag. The gateway only ends the stream; it does not kill the process, which may keep running in the sandbox. Only elapsed time tells it from a command that exits 124 on its own, so a quick exit 124 is not a timeout.

nemo_gym.sandbox.providers.openshell.provider._forget_delete(
    key: tuple[str, str],
    future: concurrent.futures.Future[None]
) -> None

nemo_gym.sandbox.providers.openshell.provider._format_conditions(
    ref: typing.Any
) -> str

The sandbox’s reported conditions as a message suffix (the gateway’s own reason, e.g. a pull failure).

nemo_gym.sandbox.providers.openshell.provider._generate_sandbox_name() -> str

Return a DNS-safe name within OpenShell’s routable-name limit.

nemo_gym.sandbox.providers.openshell.provider._grpc_status_code(
    exc: BaseException
) -> typing.Any | None

The grpc.StatusCode of an RPC error, else None.

nemo_gym.sandbox.providers.openshell.provider._is_already_exists(
    exc: BaseException
) -> bool

nemo_gym.sandbox.providers.openshell.provider._is_delete_retryable(
    e: Exception
) -> bool

Transient transport failures, or FAILED_PRECONDITION while a create’s driver operation is pending.

nemo_gym.sandbox.providers.openshell.provider._is_grpc_error(
    exc: BaseException
) -> bool

nemo_gym.sandbox.providers.openshell.provider._is_grpc_timeout(
    exc: BaseException
) -> bool

nemo_gym.sandbox.providers.openshell.provider._is_not_found(
    exc: BaseException
) -> bool

nemo_gym.sandbox.providers.openshell.provider._is_retryable_create_error(
    exc: BaseException
) -> bool

Whether a CreateSandbox RPC failure is likely transient (safe to retry with the same name).

nemo_gym.sandbox.providers.openshell.provider._is_runtime_failure(
    exc: BaseException
) -> bool

nemo_gym.sandbox.providers.openshell.provider._is_sdk_error(
    exc: BaseException
) -> bool

nemo_gym.sandbox.providers.openshell.provider._normalize_image(
    image: str
) -> str

nemo_gym.sandbox.providers.openshell.provider._phase(
    name: str
) -> int

nemo_gym.sandbox.providers.openshell.provider._phase_to_status_map() -> dict[int, nemo_gym.sandbox.providers.base.SandboxStatus]

SandboxPhase -> SandboxStatus, built from the SDK’s generated proto constants.

nemo_gym.sandbox.providers.openshell.provider._release_shared_client(
    state: nemo_gym.sandbox.providers.openshell.provider._SharedClientState
) -> None

nemo_gym.sandbox.providers.openshell.provider._require_openshell() -> None

nemo_gym.sandbox.providers.openshell.provider._wait_deleted(
    client: typing.Any,
    inst: nemo_gym.sandbox.providers.openshell.provider._OpenShellSandbox,
    operations: nemo_gym.sandbox.providers.openshell.provider.OpenShellOperationsConfig
) -> None

Poll GetSandbox until NOT_FOUND (transient RPC failures keep polling until the deadline).

nemo_gym.sandbox.providers.openshell.provider.DEFAULT_UPLOAD_CHUNK_BYTES = 512 * 1024

nemo_gym.sandbox.providers.openshell.provider.EXEC_TIMEOUT_SLACK_FRACTION = 0.1

nemo_gym.sandbox.providers.openshell.provider.EXEC_TIMEOUT_SLACK_S = 0.5

nemo_gym.sandbox.providers.openshell.provider.LOGGER = logging.getLogger(__name__)

nemo_gym.sandbox.providers.openshell.provider.MAX_DELETE_ATTEMPTS = 3

nemo_gym.sandbox.providers.openshell.provider.MAX_SANDBOX_NAME_LENGTH = 19

nemo_gym.sandbox.providers.openshell.provider.READY_PROBE_COMMAND = 'printf openshell-sandbox-ready'

nemo_gym.sandbox.providers.openshell.provider.READY_PROBE_EXPECTED = 'openshell-sandbox-ready'

nemo_gym.sandbox.providers.openshell.provider.SANDBOX_LABEL = 'nemo-gym.sandbox'

nemo_gym.sandbox.providers.openshell.provider.SANDBOX_NAME_PREFIX = 'ng-'

nemo_gym.sandbox.providers.openshell.provider.SANDBOX_NAME_RANDOM_HEX_LENGTH = MAX_SANDBOX_NAME_LENGTH - len(SANDBOX_NAME_PREFIX)

nemo_gym.sandbox.providers.openshell.provider.SANDBOX_RUNTIME_RETURN_CODE = 125

nemo_gym.sandbox.providers.openshell.provider._DELETE_STATES: dict[tuple[Any, ...], _SharedClientState] = {}

nemo_gym.sandbox.providers.openshell.provider._LIVE_SANDBOXES: set[tuple[str, str]] = set()

nemo_gym.sandbox.providers.openshell.provider._PENDING_DELETES: dict[tuple[str, str], Future[None]] = {}

nemo_gym.sandbox.providers.openshell.provider._SHARED_CLIENTS: dict[tuple[Any, ...], _SharedClientState] = {}

nemo_gym.sandbox.providers.openshell.provider._SHARED_CLIENTS_LOCK = threading.Lock()