Kubernetes

Set Up OpenShell on Kubernetes

View as Markdown

The OpenShell Helm chart is experimental and under active development. Templates, values, and defaults can change between releases. Do not use it in production.

Use the Kubernetes deployment when the gateway should run on a shared cluster, in a cloud environment, or as part of team infrastructure. The Helm chart deploys the gateway as a StatefulSet and handles PKI bootstrap, RBAC, and sandbox namespace setup automatically.

Prerequisites

Make sure the following are in place before you install.

PrerequisiteRequiredNotes
Kubernetes 1.29+ with RBAC enabledYesNo additional notes.
Helm 3.xYesNo additional notes.
Agent Sandbox controller and CRDsYesInstall before the OpenShell chart. Refer to Install Agent Sandbox.
cert-managerNoRefer to Managing Certificates. Use cert-manager only if you prefer it over the built-in PKI job.
Kubernetes Gateway APINoRefer to Ingress. Use it only for external access without port-forwarding.

Install Agent Sandbox

OpenShell uses the Agent Sandbox Kubernetes SIG project to provision sandbox pods. Install the Agent Sandbox controller and its CRDs on your cluster before installing the OpenShell Helm chart.

Fetch the latest release tag and apply the manifest:

$VERSION=$(curl -s https://api.github.com/repos/kubernetes-sigs/agent-sandbox/releases/latest | jq -r '.tag_name')
$kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/download/${VERSION}/manifest.yaml

This creates the agent-sandbox-system namespace, installs the CRDs, and starts the controller.

Confirm the controller pod is running before proceeding:

$kubectl -n agent-sandbox-system get pods

The controller pod should reach Running status within a few seconds. For cluster-specific setup instructions, including KinD and GKE walkthroughs, refer to the Agent Sandbox getting started guide.

Install OpenShell

1

Create the namespace

$kubectl create namespace openshell
2

Install the chart

Install from the OCI registry on GHCR. Replace <version> with the chart version you want to install.

$helm upgrade --install openshell \
>  oci://ghcr.io/nvidia/openshell/helm-chart \
>  --version <version> \
>  --namespace openshell

To use the latest development build instead of a stable release:

$helm upgrade --install openshell \
>  oci://ghcr.io/nvidia/openshell/helm-chart \
>  --version 0.0.0-dev \
>  --namespace openshell

The chart automatically generates PKI secrets and an SSH handshake secret on first install using pre-install Helm hooks. No manual secret creation is required.

3

Wait for the gateway to be ready

$kubectl -n openshell rollout status statefulset/openshell
4

Connect to the gateway

For local evaluation, use a port-forward:

$kubectl -n openshell port-forward svc/openshell 8080:8080

The port-forward is for local evaluation only. For shared environments, expose the gateway through your ingress controller or access proxy. Refer to Ingress for an external access option.

5

Install the client mTLS certificate

The default installation runs with mTLS enabled by default. The CLI needs the client certificate, key, and CA that the chart’s PKI hook generated. Extract them from the openshell-client-tls secret and place them where the CLI expects them for a gateway named k8s:

$mkdir -p ~/.config/openshell/gateways/k8s/mtls
$kubectl -n openshell get secret openshell-client-tls \
>  -o jsonpath='{.data.ca\.crt}'  | base64 -d > ~/.config/openshell/gateways/k8s/mtls/ca.crt
$kubectl -n openshell get secret openshell-client-tls \
>  -o jsonpath='{.data.tls\.crt}' | base64 -d > ~/.config/openshell/gateways/k8s/mtls/tls.crt
$kubectl -n openshell get secret openshell-client-tls \
>  -o jsonpath='{.data.tls\.key}' | base64 -d > ~/.config/openshell/gateways/k8s/mtls/tls.key

The server certificate SANs include localhost and 127.0.0.1, so hostname verification passes over the port-forward without extra flags.

6

Register with the CLI

In another terminal, register the gateway and verify it is reachable:

$openshell gateway add https://127.0.0.1:8080 --local --name k8s
$openshell status

Configure Chart Values

The most commonly changed values are:

ValuePurpose
image.repository / image.tagGateway container image. Defaults to ghcr.io/nvidia/openshell/gateway:latest.
server.sandboxNamespaceNamespace where sandbox pods are created. Defaults to the Helm release namespace when left empty.
server.sandboxImageDefault sandbox image used when a sandbox does not specify one.
server.grpcEndpointEndpoint that sandbox supervisors use to call back to the gateway. Must be reachable from inside the cluster.
server.sshGatewayHost / server.sshGatewayPortPublic host and port returned to CLI clients for SSH proxy connections. Required when the gateway is exposed externally.
server.disableTlsRun the gateway over plaintext HTTP. Use only behind a trusted transport.
server.enableLoopbackServiceHttpEnable local plaintext HTTP for loopback sandbox service URLs. Defaults to true.
pkiInitJob.serverDnsNames / certManager.serverDnsNamesAdditional gateway server DNS SANs. Wildcard SANs also enable sandbox service URLs under that domain.
supervisor.sideloadMethodHow the supervisor binary is delivered into sandbox pods. Leave empty to auto-detect based on cluster version: clusters running Kubernetes 1.35 or later use image-volume (ImageVolume GA in 1.36); older clusters use init-container. Set explicitly to image-volume on Kubernetes 1.33 or 1.34 with the ImageVolume feature gate enabled, or to init-container to force the legacy path on any version.

Use a values file for repeatable deployments:

$helm upgrade --install openshell \
>  oci://ghcr.io/nvidia/openshell/helm-chart \
>  --version <version> \
>  --namespace openshell \
>  --values my-values.yaml

Next Steps

  • To enable automatic certificate rotation with cert-manager, refer to Managing Certificates.
  • To expose the gateway externally without port-forwarding, refer to Ingress.
  • To configure OIDC or reverse-proxy authentication, refer to Access Control.
  • To create your first sandbox, refer to Manage Sandboxes.