Sandbox Compute Drivers
The gateway’s configured compute driver determines how OpenShell creates each sandbox. The CLI workflow stays the same across drivers: you create, connect to, inspect, and delete sandboxes through the gateway API.
Every compute driver runs the OpenShell supervisor inside the sandbox workload. The supervisor launches the agent process, applies policy, routes egress through the proxy, injects configured credentials, and maintains the gateway session.
Configure a Compute Driver
Configure the compute driver on the gateway. Current releases accept one driver per gateway. Set compute_drivers in the gateway TOML file:
Reserved built-in values are docker, podman, kubernetes, and vm.
Non-reserved names select an extension driver and require a
socket_path in [openshell.drivers.<name>].
When compute_drivers is unset, the gateway auto-detects Kubernetes, then Podman, then Docker. Local container runtimes must respond to an API probe before the gateway selects them. The VM driver is never auto-detected; configure it explicitly with compute_drivers = ["vm"] or set OPENSHELL_DRIVERS=vm in the launch environment.
Common gateway options:
Set driver-specific values such as sandbox images, callback endpoints, network names, TLS material, and VM sizing in the gateway TOML file. See the Gateway Configuration File reference for the full [openshell.drivers.<name>] schema.
Extension drivers use the same compute_driver.proto gRPC surface as the
managed VM driver. For an out-of-tree driver, choose a driver name and point
the gateway at the Unix socket the operator has already provisioned:
For a launch-time socket override, pass the same non-reserved driver name with the socket path:
The gateway does not spawn, supervise, or delete extension drivers. The operator must protect the socket so only the gateway uid can access it. Reserved built-in names cannot be selected through unmanaged socket endpoints.
Sandbox create supports --cpu and --memory for per-sandbox compute sizing.
Docker and Podman apply them as runtime limits. Kubernetes applies them as both
container requests and limits. The VM driver accepts the fields but currently
ignores them.
Sandbox create also accepts experimental driver-owned config through
--driver-config-json. The value is a JSON object keyed by driver name. The
gateway forwards only the block for the active driver, so a Kubernetes gateway
receives the kubernetes object from a value such as:
Nested keys inside each driver block use snake_case. The top-level envelope keys
are driver names, such as kubernetes, and are not part of the nested schema.
Driver config is for fields without a stable public flag. Prefer --cpu,
--memory, and --gpu for supported resource intent. When --gpu is present
without a count, OpenShell treats it as a request for one GPU. Pass
--gpu COUNT when requesting more than one GPU.
Kubernetes maps the GPU count to the nvidia.com/gpu pod resource limit.
Docker and Podman satisfy count-only GPU requests by selecting the requested
number of NVIDIA CDI devices from the local CDI inventory in round-robin order.
The drivers refresh the CDI inventory before validating or creating the
sandbox, so CDI devices added or removed after gateway start can affect later
creates. On WSL2 all-only runtimes, Docker or Podman can use
nvidia.com/gpu=all as a compatibility fallback, where it counts as one
selectable device.
Exact GPU device selection remains driver-owned and requires --gpu. Docker
and Podman accept cdi_devices as opaque CDI device names; replace the
top-level docker key with podman when using the Podman driver, for example
{"docker":{"cdi_devices":["nvidia.com/gpu=0"]}}. Explicit CDI device lists
must not contain duplicates, and their length must match the effective GPU
count. A single exact CDI device is compatible with the default --gpu
request. The VM driver accepts gpu_device_ids, for example
{"vm":{"gpu_device_ids":["0000:2d:00.0"]}}; the current VM implementation
accepts at most one entry and allows either --gpu or --gpu 1 when
gpu_device_ids is set.
For Kubernetes, pod.runtime_class_name maps to PodSpec runtimeClassName.
It overrides the gateway’s configured default runtime class for that sandbox,
while a typed SandboxTemplate.runtime_class_name value from the API still
takes precedence.
Docker and Podman callback listeners accept only supervisor callback gRPC
methods. Use the gateway’s primary endpoint for CLI, administrator, health,
reflection, inference-route management, and HTTP requests. A
PermissionDenied response from one of the sandbox-visible callback addresses
is expected for those requests. The gateway fails startup if a callback
requirement resolves to the exact primary listener address because one socket
cannot preserve both authorization scopes. For the IPv4-loopback callback used
by Podman Machine, set bind_address = "[::1]:17670" for the primary listener
and register https://localhost:17670 as the CLI endpoint. The hostname matches
the generated certificate and avoids the TLS transport error produced by a raw
IPv6-literal endpoint. Do not broaden the primary listener to 0.0.0.0.
Docker Driver
Docker-backed sandboxes run as containers on the gateway host. Use Docker for local development, single-machine gateways, and hosts that already use Docker Desktop or Docker Engine.
The gateway talks to the Docker daemon to create sandbox containers. Docker is also required for local image builds from directories or Dockerfiles.
For maintainer-level implementation details, refer to the Docker driver README.
Select Docker with compute_drivers = ["docker"] in [openshell.gateway]. Configure Docker driver values such as socket_path, grpc_endpoint, network_name, supervisor_bin, supervisor_image, image_pull_policy, ssh_socket_path, sandbox_pids_limit, and guest_tls_* in [openshell.drivers.docker]. When socket_path is unset, the driver uses the same responsive local socket selected by auto-detection. An explicitly selected Docker driver falls back to /var/run/docker.sock when no candidate responds.
For GPU-backed Docker sandboxes, configure Docker CDI before starting the gateway so OpenShell can detect the daemon capability.
Docker Driver Config Mounts
Docker driver config accepts user-supplied volume and tmpfs mounts. It also
accepts bind mounts when [openshell.drivers.docker] sets
enable_bind_mounts = true in gateway.toml. See Docker’s storage documentation for more information.
Docker local-driver named volumes created with bind options also expose
gateway-host paths, so OpenShell treats them like bind mounts and requires
enable_bind_mounts = true.
Use a volume mount for existing Docker named volumes:
Bind mounts share gateway-host filesystem resources with the sandbox. They may be considered insecure because they can negate OpenShell controls such as workspace isolation and filesystem policy. Use them only when you understand and accept that loss of isolation.
Use a bind mount only after enabling it in the Docker driver table:
Docker mount schema:
OpenShell rejects mount source, target, and Docker volume subpath values
with surrounding whitespace. OpenShell also rejects mount targets that replace
the workspace root or container root, or contain or are contained by the
configured SSH socket or reserved /opt/openshell, /etc/openshell,
/etc/openshell-tls, /run/openshell, /run/openshell-sidecar, and network
namespace roots. These checks do not make host bind mounts safe.
Podman Driver
Podman-backed sandboxes run as rootless containers on the gateway host. Use Podman for Linux workstation workflows that avoid a rootful Docker daemon.
The gateway talks to the Podman API socket. The Podman driver requires Podman 5.x, cgroups v2, rootless networking, and an active Podman user socket. When socket_path is not set, the driver probes for a responsive Podman socket and fails to start if none respond.
For maintainer-level implementation details, refer to the Podman driver README and Podman networking notes.
Select Podman with compute_drivers = ["podman"] in [openshell.gateway]. Configure Podman driver values such as socket_path, network_name, supervisor_image, stop_timeout_secs, image_pull_policy, grpc_endpoint, host_gateway_ip, sandbox_ssh_socket_path, sandbox_pids_limit, and guest_tls_* in [openshell.drivers.podman].
Podman sandboxes default to a 45-second graceful stop window before Podman escalates from SIGTERM to SIGKILL. Set stop_timeout_secs in gateway config, or OPENSHELL_STOP_TIMEOUT for the standalone driver, when a local runtime needs a different teardown window.
For proxy-required networks, the Podman driver also accepts the corporate egress proxy keys https_proxy, no_proxy, proxy_auth_file, proxy_auth_allow_insecure, and proxy_connect_by_hostname. The supervisor chains policy-approved TLS tunnels through the proxy with HTTP CONNECT instead of dialing destinations directly. See the Gateway Configuration File reference for the full contract, including the cleartext-credential acknowledgement and the validated-IP CONNECT behavior.
On macOS with podman machine, the driver uses gvproxy’s host-loopback IP, 192.168.127.254, for sandbox host aliases by default. Set host_gateway_ip only when your Podman machine uses a non-standard host-loopback address. On Linux, an empty host_gateway_ip keeps Podman’s host-gateway resolver behavior. Direct local callbacks from rootless Podman require Podman to report the pasta network helper. Slirp4netns, other helpers, and Podman versions that do not report their helper require an explicitly remote grpc_endpoint; otherwise the gateway fails startup rather than leaving sandbox callbacks unreachable. Rootful Podman continues to use the configured network’s bridge gateway address.
Podman Driver Config Mounts
Podman driver config accepts user-supplied volume, tmpfs, and image
mounts. It also accepts bind mounts when [openshell.drivers.podman] sets
enable_bind_mounts = true in gateway.toml. Podman local-driver named
volumes created with bind options also expose gateway-host paths, so
OpenShell treats them like bind mounts and requires enable_bind_mounts = true.
Host bind mounts expose gateway host paths to sandbox requests, so they are
disabled by default.
Use a volume mount for existing Podman named volumes:
Bind mounts share gateway-host filesystem resources with the sandbox. They may be considered insecure because they can negate OpenShell controls such as workspace isolation and filesystem policy. Use them only when you understand and accept that loss of isolation.
Use a bind mount only after enabling it in the Podman driver table:
Podman mount schema:
Podman volume and image mounts do not support subpath in OpenShell driver
config, and OpenShell rejects subpath for those mount types. OpenShell rejects
mount source and target values with surrounding whitespace. OpenShell also
rejects mount targets that replace the workspace root, container root, supervisor
files, /etc/openshell, /etc/openshell-tls, authentication material, or
network namespace paths. These checks do not make host bind mounts safe.
MicroVM Driver
MicroVM-backed sandboxes run inside VM-backed isolation instead of a container boundary. Use MicroVM when workloads need a VM boundary instead of a local container boundary.
The gateway uses the VM compute driver to create VM-backed sandboxes. MicroVM requires host virtualization support. It uses libkrun with Apple’s Hypervisor framework on macOS, KVM on Linux, and QEMU for GPU-backed sandboxes on Linux.
The VM driver boots a cached immutable bootstrap ext4 root disk. When the requested sandbox image differs from the bootstrap image, the driver stages the registry image as an OCI layout, unpacks it inside a bootstrap VM with umoci, and caches the prepared image disk by image identity. Each sandbox receives that prepared disk read-only plus its own writable overlay.ext4 disk for /, including /sandbox writes and runtime TLS material. The overlay persists for the sandbox lifetime and is deleted with the sandbox state directory.
VM sandbox creation follows the same progress model as Kubernetes-backed sandboxes. The gateway accepts the sandbox, then the VM driver publishes watch events while it resolves the image, prepares or reuses the bootstrap and prepared image caches, creates the writable overlay, and starts the VM launcher.
On gateway restart, the gateway starts a fresh VM driver process. The driver scans its state directory for accepted sandbox launch records, restarts those VMs, and reuses each sandbox’s existing overlay.ext4 so files written inside the sandbox remain available after the supervisor reconnects.
For maintainer-level implementation details, refer to the VM driver README.
Enable the VM Driver
The VM driver is opt-in. Release packages can install openshell-driver-vm, but the gateway does not select it unless you configure the driver explicitly.
Enable VM by setting compute_drivers = ["vm"] in the gateway TOML file:
For a launch-time override, set OPENSHELL_DRIVERS=vm in the gateway environment and restart the service.
Configure VM driver values such as grpc_endpoint, driver_dir, state_dir, default_image, bootstrap_image, vcpus, mem_mib, overlay_disk_mib, krun_log_level, and guest_tls_* in [openshell.drivers.vm]. The VM state_dir stores overlay disks, console logs, runtime state, image-rootfs cache, and the private run/compute-driver.sock socket. The VM socket path is managed by the gateway and is not configurable through remote endpoint settings.
The gateway starts openshell-driver-vm over a private Unix socket and passes its process ID so the driver can reject unexpected local clients. The driver’s standalone TCP listener is disabled unless --allow-unauthenticated-tcp is set for local development.
Local image resolution
The VM driver resolves sandbox images from a local container engine before falling back to registry pulls. It tries Docker first, then falls back to the Podman socket (Docker-compatible API). On Linux with Podman, enable the API socket so the driver can find local images:
Host Firewall
The VM driver creates nftables rules on the host for each sandbox VM’s TAP network interface. These rules provide NAT for VM connectivity and defense-in-depth isolation: unsolicited inbound connections to the VM are dropped, and the VM can only reach the gateway port on the host. Primary security enforcement (proxy-only egress and bypass detection) is handled by the sandbox supervisor inside the VM guest.
On hosts with restrictive firewalls (e.g. firewalld), the host firewall may additionally block VM traffic that the driver’s rules accept. If VM sandboxes cannot reach the network, verify that the host firewall allows forwarding and input for vmtap-* interfaces. See the VM driver README for details.
Kubernetes Driver
Kubernetes-backed sandboxes run as pods in the configured sandbox namespace. Use Kubernetes for shared clusters, remote compute, GPU scheduling, and operator-managed environments.
Helm deployments set Kubernetes driver values through the chart.
For maintainer-level implementation details, refer to the Kubernetes driver README.
In combined topology, the agent container carries the Linux capabilities
needed by the supervisor for network namespace setup, Landlock filesystem
policy, process privilege changes, and network policy enforcement. In sidecar
topology, the agent container runs as the resolved sandbox UID/GID with no added
Linux capabilities. A root init container performs the nftables setup, and the
long-running binary-aware sidecar runs as UID 0, drops default capabilities,
and adds SYS_PTRACE plus DAC_READ_SEARCH for workload process identity
resolution through shared /proc. The
sidecar.process_binary_aware_network_policy = false setting runs it as the
configured non-root proxy_uid, removes both capabilities, and relaxes network
policy to endpoint/L7 matching only. The
network sidecar owns gateway authentication and writes local policy/provider
state to the process supervisor over a local control socket, so the agent
container does not mount the sandbox bootstrap token or client TLS secret in
the default sidecar path. The provider environment is refreshed by the network
sidecar after settings polls and streamed to the process supervisor so future
child processes can see updated provider env without gateway access in the
agent container.
Sidecar mode keeps gateway session and SSH behavior. The process supervisor
applies Landlock filesystem policy and child seccomp filters where supported,
but it does not perform root-to-sandbox privilege dropping or supervisor
identity mount isolation. Network policy still runs in the sidecar, and sidecar
pods set shareProcessNamespace: true so the network sidecar can resolve
process/binary identity through /proc/<entrypoint-pid>.
The Kubernetes driver creates namespaced agents.x-k8s.io Sandbox resources from the Kubernetes SIG Apps agent-sandbox project. It detects the served Sandbox API at runtime, caches the selected API version for the gateway process, and uses v1beta1 when available before falling back to v1alpha1, so supported Agent Sandbox installations work without version-specific operator configuration. The Agent Sandbox controller turns those resources into sandbox pods and related storage.
If Agent Sandbox is upgraded in place, restart the OpenShell gateway after the controller and CRD rollout completes so the gateway can detect the served API versions again.
Sandbox.spec.volumeClaimTemplates is immutable after creation. To change storage configuration, delete the sandbox and create a new one with the updated spec.
Kubernetes Driver Config PVC Mounts
Kubernetes driver config can mount existing PersistentVolumeClaims into the
agent container. Use this when storage is provisioned outside OpenShell and a
sandbox should mount selected PVC subpaths instead of using the default
OpenShell-created /sandbox workspace PVC.
Kubernetes PVC mount schema:
OpenShell rejects duplicate volume names, mounts that reference unknown volumes,
protected mount targets, and mounts that replace OpenShell TLS, supervisor,
ServiceAccount token, or SPIFFE paths. Read-write PVC access requires
read_only: false on both the PVC volume and each writable mount.
Any driver-config mount under /sandbox disables the default /sandbox
workspace PVC injection for that sandbox. Only the explicit mount paths persist
through the external PVC; other /sandbox paths come from the current sandbox
image.
Sandbox User Identity
The policy can set process.run_as_user and process.run_as_group
independently. Each explicit field wins. The active compute driver supplies the
identity for omitted fields.
Docker / Podman
Docker and Podman inspect the final image and use its OCI USER declaration as
a per-field fallback. Supported forms include app, app:staff, a numeric UID
whose passwd entry supplies its primary GID, and an accountless numeric pair
such as 1234:1235.
The driver pins container creation to the immutable image ID it inspected. The
supervisor validates any required names inside that image and preserves the
declared name or numeric components for both direct and SSH children. When
USER omits the group, the supervisor uses the user’s numeric primary GID. It
does not modify /etc/passwd or /etc/group.
Docker also inspects OCI WorkingDir. An absolute value becomes the
agent workspace; an empty, root (/), or explicit /sandbox value uses the
managed /sandbox compatibility workspace.
OpenShell creates and owns that compatibility workspace. Any other workdir must
already exist in the immutable image without symlink components. The completed
UID/GID and supplementary groups must already be able to traverse every parent
and write and enter the workdir. OpenShell does not change that directory’s
ownership or mode. A one-shot validator drops to that identity and uses kernel
effective-access checks, including POSIX ACL grants and LSM denials. It rejects
workdirs that overlap the OCI runtime namespaces under /proc, /sys, or
/dev, and rejects overlap with actual OpenShell control paths. Docker checks
the original image filesystem in the final supervisor and rejects image
VOLUME declarations that would mask the workdir or one of its parents before
validation. The resolved workspace is the cwd and HOME for direct and SSH
children. The supervisor itself starts from /, so a missing or invalid
workspace is handled during readiness instead of preventing the container
runtime from starting it.
Sandbox creation fails before readiness if a required USER component is
missing, malformed, unknown, ambiguous, or resolves to UID/GID 0. An image
without USER therefore works only when policy explicitly provides both
identity fields.
Kubernetes / OpenShift
The Kubernetes driver auto-detects the sandbox UID from OpenShift SCC namespace annotations:
openshift.io/sa.scc.uid-range(format:<start>/<size>, e.g.1000000000/10000) provides the UID.openshift.io/sa.scc.supplemental-groupsprovides the GID when present; otherwise the resolved UID is used as the GID.- On non-OpenShift clusters, or when annotations are absent, the driver falls back to
1000.
You can override autodetection with explicit sandbox_uid / sandbox_gid config in [openshell.drivers.kubernetes]. When set, the driver skips namespace annotation lookup entirely.
The resolved UID/GID appear in:
- Supervisor container environment variables (
OPENSHELL_SANDBOX_UID,OPENSHELL_SANDBOX_GID) for direct kernel-level privilege dropping without/etc/passwdlookups. - PVC init container
securityContext.runAsUser/runAsGroup/fsGroupfor workspace ownership operations.
VM Driver
The VM driver injects the sandbox UID into the rootfs guest’s /etc/passwd, /etc/group, and /etc/gshadow during rootfs preparation. Default UID is 10001; configure sandbox_uid in [openshell.drivers.vm] to use a different value.
Custom Images
Docker and Podman custom images do not need a baked-in "sandbox" user. Declare
a non-root OCI USER, or set both process identity fields explicitly in policy.
Named image users require matching account entries; a numeric UID:GID pair
does not. For Docker, declare an absolute OCI WORKDIR to select the workspace.
Images with no working directory, WORKDIR /, or WORKDIR /sandbox use
OpenShell’s managed /sandbox compatibility workspace. For any other Docker
path, create the directory in the image and grant the final process identity
write and execute permission in the Dockerfile. Podman, Kubernetes/OpenShift,
and VM sandboxes continue to use /sandbox.