Set Up OpenShell on Kubernetes

The gateway’s Kubernetes compute driver creates a Sandbox resource and a separate supervisor Pod. The Agent Sandbox controller creates the workload Pod from that resource. A Service gives the supervisor a stable address for the workload’s authenticated TLS boundary, and a PVC preserves /sandbox across Pod restarts. The gateway can run in the same namespace as these resources or in a separate namespace.

The supervisor opens the boundary connection and maintains an outbound session to the gateway. A NetworkPolicy allows only supervisor ingress to the workload’s boundary port and denies new workload-initiated connections. Agent network requests return on the established channel; the supervisor evaluates policy and opens approved upstream connections. For the trust boundaries and request flow, see Architecture.

Prerequisites

Make sure the following are in place before you install.

Install Agent Sandbox

OpenShell uses the Agent Sandbox Kubernetes SIG project to provision sandbox pods. Install the Agent Sandbox controller and its CRDs on your cluster before installing the OpenShell Helm chart.

Apply the latest release manifest:

kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/latest/download/sandbox.yaml

This creates the agent-sandbox-system namespace, installs the sandboxes.agents.x-k8s.io CRD, and starts the controller.

The Helm chart checks for a supported Agent Sandbox API before it creates gateway resources. This preflight is enabled by default. Disable it only for offline helm template rendering, where Helm cannot discover cluster APIs:

helm template openshell oci://ghcr.io/nvidia/openshell/helm-chart \
  --version <version> \
  --set agentSandbox.preflight.enabled=false

The chart does not install or upgrade the cluster-scoped Agent Sandbox CRDs or controller.

Confirm the controller pod is running before proceeding:

kubectl -n agent-sandbox-system get pods

The controller pod should reach Running status within a few seconds. For cluster-specific setup instructions, including KinD and GKE walkthroughs, refer to the Agent Sandbox getting started guide.

Upgrade Agent Sandbox

OpenShell detects the served Agent Sandbox Sandbox API when the Kubernetes gateway first needs it and caches that choice for the gateway process. If you upgrade Agent Sandbox in place, restart the OpenShell gateway after the Agent Sandbox controller and CRD rollout completes so the gateway can detect the served API versions again. Existing sandboxes keep running during the upgrade, and the restarted gateway can continue managing them.

Install OpenShell

kubectl create namespace openshell

helm upgrade --install openshell \
  oci://ghcr.io/nvidia/openshell/helm-chart \
  --version <version> \
  --namespace openshell

helm upgrade --install openshell \
  oci://ghcr.io/nvidia/openshell/helm-chart \
  --version 0.0.0-dev \
  --namespace openshell

helm upgrade --install openshell \
  oci://ghcr.io/nvidia/openshell/helm-chart \
  --version <version> \
  --namespace openshell \
  --set workspaceResources.enabled=false \
  --set server.sandboxNamespace=app-a
helm upgrade --install openshell-workspace \
  oci://ghcr.io/nvidia/openshell/openshell-workspace \
  --version <version> \
  --namespace app-a \
  --set gateway.serviceAccount.name=openshell \
  --set gateway.serviceAccount.namespace=openshell

helm upgrade --install openshell \
  oci://ghcr.io/nvidia/openshell/helm-chart \
  --version <version> \
  --namespace openshell \
  --set server.credentialDrivers.kubernetesSecrets.enabled=true \
  --set server.credentialDrivers.kubernetesSecrets.namespace=openshell-credentials \
  --set server.credentialDrivers.kubernetesSecrets.createNamespace=true

kubectl -n openshell rollout status statefulset/openshell

kubectl -n openshell rollout status deployment/openshell

kubectl -n openshell port-forward svc/openshell 8080:8080

mkdir -p ~/.config/openshell/gateways/k8s/mtls
kubectl -n openshell get secret openshell-client-tls \
  -o jsonpath='{.data.ca\.crt}'  | base64 -d > ~/.config/openshell/gateways/k8s/mtls/ca.crt
kubectl -n openshell get secret openshell-client-tls \
  -o jsonpath='{.data.tls\.crt}' | base64 -d > ~/.config/openshell/gateways/k8s/mtls/tls.crt
kubectl -n openshell get secret openshell-client-tls \
  -o jsonpath='{.data.tls\.key}' | base64 -d > ~/.config/openshell/gateways/k8s/mtls/tls.key

openshell gateway add https://127.0.0.1:8080 --local --name k8s \
  --oidc-issuer https://your-idp.example.com/realms/openshell \
  --oidc-client-id openshell-cli
openshell status

Configure Chart Values

The Helm chart renders gatewayConfig into the mounted gateway.toml ConfigMap. Each top-level key names a TOML table, and its fields use the names in the gateway configuration reference. For example, set the gateway log level with:

helm upgrade openshell \
  oci://ghcr.io/nvidia/openshell/helm-chart \
  --version <version> \
  --namespace openshell \
  --set 'gatewayConfig.openshell\.gateway.log_level=debug'

Keep credentials, database URLs, and private keys in Kubernetes Secrets, using the chart’s environment-variable, file, or volume wiring. The chart rejects database_url, inline URL credentials, and PEM private keys in gatewayConfig, but it does not detect every possible secret.

Helm values for Services, probes, TLS certificates, Secret references, RBAC, NetworkPolicies, and mounts still control those Kubernetes resources. The chart derives their corresponding runtime settings to keep the gateway configuration aligned with the deployed resources.

YAML scalar values retain their types, scalar lists become TOML arrays, and nested maps become inline tables. Top-level lists become arrays of tables and preserve item order. Null fields are omitted; null array elements are invalid. Helm template expressions in strings are not evaluated.

Existing application values remain supported as deprecated aliases. Explicit gatewayConfig fields take precedence, including resource-admission settings and the Kubernetes Secrets credential namespace. The global image pull policy applies to runtime and supervisor images. Leave image_pull_policy and sandbox.image.pullPolicy unset to use Kubernetes defaults for the workload image.

The most commonly changed values are:

Use a values file for repeatable deployments:

helm upgrade --install openshell \
  oci://ghcr.io/nvidia/openshell/helm-chart \
  --version <version> \
  --namespace openshell \
  --values my-values.yaml

To use private sandbox images, create a kubernetes.io/dockerconfigjson Secret in the sandbox namespace and reference its name:

kubectl -n openshell create secret docker-registry regcred \
  --docker-server=registry.example.com \
  --docker-username="$REGISTRY_USER" \
  --docker-password="$REGISTRY_TOKEN"

server:
  sandboxImage: registry.example.com/team/openshell-sandbox:latest
  sandboxImagePullSecrets:
    - name: regcred

Configure a Corporate Upstream Proxy

Configure a corporate forward proxy when sandbox TLS egress cannot dial the Internet directly. OpenShell evaluates policy and SSRF checks before it opens an HTTP CONNECT tunnel through the proxy. The proxy URL is operator-owned configuration. Sandbox environment variables cannot select, replace, or bypass it.

Create the credential Secret in the sandbox namespace when the proxy requires Basic authentication. The Secret value uses the user:pass form.

kubectl -n openshell create secret generic corporate-proxy-auth \
  --from-literal=credentials="$PROXY_USER:$PROXY_PASSWORD"

Add the proxy settings to your Helm values file. Replace the DNS suffixes and CIDRs in noProxy with values for your cluster. noProxy bypasses only the corporate proxy. OpenShell policy evaluation still applies.

upstreamProxy:
  url: http://proxy.corp.example:8080
  noProxy: .svc,.svc.cluster.local,10.96.0.0/12,10.244.0.0/16
  authSecret:
    name: corporate-proxy-auth
    key: credentials
  authAllowInsecure: true

Use authAllowInsecure: true only when you accept that Basic authentication is cleartext on the connection to an http:// proxy. The initial release supports http:// proxy endpoints and TLS CONNECT egress. It does not support HTTPS-to-proxy, custom corporate CA bundles, or forwarding plain HTTP egress through the proxy.

The credential mounts only in the separately scheduled supervisor Pod. The sandbox workload cannot read it through its environment or volumes.

RBAC

The chart creates the following RBAC resources in the release namespace:

Every other object the chart creates is namespaced. The ClusterRole and ClusterRoleBinding in the last row are the only cluster-scoped objects.

When the Kubernetes Secrets credential driver is enabled, the chart also creates an openshell-credential-secrets Role and RoleBinding in the credential namespace. That Role grants get, create, patch, and delete on all Secrets in the credential namespace, and is the gateway’s only permission grant for provider credential Secrets.

In managed and operator workspace modes, the chart creates an openshell-workspace-secret-source Role and RoleBinding in the sandbox namespace. That Role grants get on only the gateway client TLS Secret and, in managed mode, the configured image-pull Secrets. The gateway stages their contents into each sandbox runtime generation’s Secrets in the workspace namespace.

The namespaced Role covers sandbox lifecycle and identity:

The ClusterRole grants node inspection and token validation:

To use an existing ServiceAccount instead of creating one, set serviceAccount.create=false and supply its name:

helm upgrade --install openshell \
  oci://ghcr.io/nvidia/openshell/helm-chart \
  --version <version> \
  --namespace openshell \
  --set serviceAccount.create=false \
  --set serviceAccount.name=my-existing-sa

The ServiceAccount must already have the Role and ClusterRole bindings described above.

Installing without cluster-admin

By default the release creates the ClusterRole and ClusterRoleBinding, so the installer needs cluster-scoped permissions. When cluster-scoped RBAC is owned by a different team, or the installer is a namespace-admin GitOps controller, split the install into a cluster-admin step and a namespace-admin step.

A cluster-admin applies the cluster-scoped objects once per gateway ServiceAccount. Render them from the same values the release uses, so the rules match the configured workspace mode and credential driver:

helm template openshell \
  oci://ghcr.io/nvidia/openshell/helm-chart \
  --version <version> \
  --namespace openshell \
  -f my-values.yaml \
  --set rbac.create=true \
  --set rbac.clusterScoped.create=true \
  --set agentSandbox.preflight.enabled=false \
  --show-only templates/clusterrole.yaml \
  --show-only templates/clusterrolebinding.yaml | kubectl apply -f -

A namespace-admin then installs and upgrades the chart with cluster-scoped objects omitted. The release renders only namespaced objects and needs no permission on clusterroles or clusterrolebindings:

helm upgrade --install openshell \
  oci://ghcr.io/nvidia/openshell/helm-chart \
  --version <version> \
  --namespace openshell \
  -f my-values.yaml \
  --set rbac.clusterScoped.create=false

The gateway ServiceAccount name and namespace are unchanged by this flag, so the pre-created ClusterRoleBinding still binds the ServiceAccount the namespaced release creates. The same holds with serviceAccount.create=false: the binding subject follows serviceAccount.name, so pass the same values to both steps.

Migrating an existing release

Helm deletes objects that leave a release manifest, so setting rbac.clusterScoped.create=false on a release that already owns the ClusterRole and ClusterRoleBinding deletes them. The gateway then loses TokenReview until a cluster-admin re-applies them. Hand ownership over first, as cluster-admin, so nothing is deleted:

kubectl annotate clusterrole "openshell-node-reader-<namespace>" \
  helm.sh/resource-policy=keep --overwrite
kubectl annotate clusterrolebinding "openshell-node-reader-<namespace>" \
  helm.sh/resource-policy=keep --overwrite

The objects then survive the upgrade that sets the flag, and the cluster-admin owns them from that point on. Fresh installs need no such step.

rbac.clusterScoped.create is independent of the workspace mode. managed and operator modes change what the ClusterRole grants, but they never require a namespace-admin to apply it. Re-run the cluster-admin step after changing any value that affects the ClusterRole rules.

Set rbac.clusterScoped.clusterRoleName and rbac.clusterScoped.clusterRoleBindingName when the cluster-admin owns the naming.

Which flag the installer needs

In the default shared workspace mode the release also creates a namespaced sandbox Role granting Agent Sandbox (agents.x-k8s.io) permissions. Kubernetes forbids granting permissions you do not hold, and the built-in admin ClusterRole does not cover that CRD, so an installer holding only admin cannot create it. rbac.clusterScoped.create=false alone is then not enough, and the install fails with attempting to grant RBAC permissions not currently held.

managed and operator render no namespaced sandbox Role, so no extra grant is needed there.

With rbac.create=false, the cluster-admin applies the namespaced RBAC in the same step by adding it to the render:

helm template openshell \
  oci://ghcr.io/nvidia/openshell/helm-chart \
  --version <version> \
  --namespace openshell \
  -f my-values.yaml \
  --set rbac.create=true \
  --set rbac.clusterScoped.create=true \
  --set agentSandbox.preflight.enabled=false \
  --show-only templates/clusterrole.yaml \
  --show-only templates/clusterrolebinding.yaml \
  --show-only templates/role.yaml \
  --show-only templates/rolebinding.yaml \
  --show-only templates/peer-role.yaml | kubectl apply -f -

To grant the installer the sandbox permissions instead, bind it to a Role carrying the same rules as the chart’s openshell-sandbox Role.

Probes

The gateway exposes /healthz for process liveness and /readyz for dependency-aware readiness on the health port. The Helm chart wires both into Kubernetes probes:

• startupProbe and livenessProbe use /healthz.
• readinessProbe uses /readyz, which reflects the latest result of an in-process background database check.

Each sandbox supervisor Pod uses a tcpSocket readiness probe on port 5501. The supervisor accepts connections on that port only while its gateway session is ready. If you add a default-deny ingress policy to sandbox namespaces, allow TCP port 5501 from the node so kubelet can reach supervisor Pods.

Next Steps

• To run multiple gateway replicas, refer to High Availability.
• To enable automatic certificate rotation with cert-manager, refer to Managing Certificates.
• To expose the gateway externally without port-forwarding, refer to Ingress.
• To configure OIDC or reverse-proxy authentication, refer to Access Control.
• To create your first sandbox, refer to Manage Sandboxes.