OpenSandbox

The NeMo Platform Helm chart does not install OpenSandbox. Install the upstream charts, then point the platform at the running server.

Install OpenSandbox when you run sandboxed GRPO / NeMo Gym: untrusted custom environment FileSets run in isolated pods, not in the training container. It does not sandbox the rest of the platform (API, DPO, SFT, inference).

This page is the shared-kernel path: sandbox pods use the cluster default OCI runtime. That is often runc (containerd) or crun (CRI-O, including OKE and OpenShift). A kernel-isolated runtime is not required. Use OpenSandbox with Kata when those Gym/GRPO sandbox pods must be isolated from the host kernel. The documented path is Kata QEMU because it gives each sandbox its own guest kernel; other isolated runtimes may work but have not been tested. Example Helm values live in k8s/helm/examples/opensandbox/.

OpenSandbox [secure_runtime] is server-global. One OpenSandbox server cannot mix shared-kernel and Kata, so point the platform at one Service DNS. The cluster keeps its default OCI runtime for every other workload.

Upstream Helm, TOML, and SDK documentation: Kubernetes deployment, configuration, server configuration reference.

Prerequisites

• A local OpenSandbox checkout with Helm charts under kubernetes/charts/, or a published chart tarball
• kubectl access to the cluster
• The NeMo Platform Helm release namespace (jobs run here; it is also [kubernetes] namespace in the server values)

Namespace rule

OpenSandbox config.toml [kubernetes] namespace must be the same namespace the platform jobs run in (the Helm release namespace). Control-plane Deployments, Services, and template ConfigMaps can stay in opensandbox-system. Sandbox pods remount the job-storage PVC by claim name and resolve image-pull secrets in their own namespace, so a dedicated opensandbox workload namespace will fail PVC remounts and image pulls.

Values jobs need

Training and other job pods are OpenSandbox clients. They do not need a kubeconfig to the sandbox API. The Python SDK builds http(s)://{domain}/v1 from:

Gym sample YAML uses OPENSANDBOX_DOMAIN / OPENSANDBOX_API_KEY. NeMo Platform and the SDK use OPEN_SANDBOX_*. Do not mix the names.

Install the shared-kernel server

Replace NMP_NAMESPACE with the platform release namespace.

1
export NMP_NAMESPACE=nemo-platform
2
export OPENSANDBOX_DIR=/path/to/OpenSandbox
3
export EXAMPLES=k8s/helm/examples/opensandbox
4
5
sed -i.bak "s/REPLACE_WITH_RELEASE_NAMESPACE/${NMP_NAMESPACE}/g" \
6
  "${EXAMPLES}/opensandbox-server.yaml"
7
8
kubectl create namespace opensandbox-system --dry-run=client -o yaml | kubectl apply -f -
9
kubectl create namespace "$NMP_NAMESPACE" --dry-run=client -o yaml | kubectl apply -f -
10
11
kubectl apply -f "${EXAMPLES}/batchsandbox-template.yaml"
12
13
kubectl create secret generic opensandbox-server-api-key \
14
  -n opensandbox-system --from-literal=api-key="$(openssl rand -hex 32)"
15
16
# Jobs cannot secretKeyRef across namespaces. Copy the key into the job namespace.
17
kubectl get secret opensandbox-server-api-key -n opensandbox-system -o json \
18
  | jq 'del(.metadata.uid,.metadata.resourceVersion,.metadata.creationTimestamp,.metadata.namespace)' \
19
  | kubectl apply -n "$NMP_NAMESPACE" -f -
20
21
# The BatchSandbox template hard-codes imagePullSecrets.name: nvcrimagepullsecret.
22
# That Secret must exist in $NMP_NAMESPACE. If yours has a different name, edit
23
# imagePullSecrets in batchsandbox-template.yaml (and batchsandbox-template-kata-qemu.yaml
24
# if you use Kata) before kubectl apply.
25
kubectl get secret nvcrimagepullsecret -n "$NMP_NAMESPACE"
26
27
helm upgrade --install opensandbox-controller \
28
  "${OPENSANDBOX_DIR}/kubernetes/charts/opensandbox-controller" \
29
  --namespace opensandbox-system \
30
  -f "${EXAMPLES}/opensandbox-controller.yaml"
31
32
helm upgrade --install opensandbox-server \
33
  "${OPENSANDBOX_DIR}/kubernetes/charts/opensandbox-server" \
34
  --namespace opensandbox-system \
35
  -f "${EXAMPLES}/opensandbox-server.yaml"

Point the platform at the server

1
sandboxClusterCapable: true
2
opensandbox:
3
  domain: opensandbox-server.opensandbox-system.svc.cluster.local
4
  protocol: http
5
  apiKeySecret: opensandbox-server-api-key
6
  apiKeySecretKey: api-key

When sandboxClusterCapable is true, Kubernetes and Volcano job pods receive OPEN_SANDBOX_DOMAIN and a Secret-backed OPEN_SANDBOX_API_KEY. The chart still does not create OpenSandbox Deployments.

Verify

1
export OPEN_SANDBOX_WORKLOAD_NS="$NMP_NAMESPACE"
2
./k8s/helm/examples/opensandbox/verify/shared-kernel.sh

The verifier checks the server Deployment, /health, creates a short-lived sandbox in the job namespace, and asserts an empty runtimeClassName (cluster default OCI runtime).