Kubernetes

Ingress

View as Markdown

By default, the OpenShell gateway is only reachable inside the cluster. To let CLI clients connect without a kubectl port-forward, expose the gateway through an ingress.

OpenShell uses the Kubernetes Gateway API for ingress. The chart creates a GRPCRoute that routes inbound gRPC traffic to the gateway pod. You need a Gateway API implementation installed on your cluster to fulfill the GRPCRoute. This page uses Envoy Gateway, which the chart is tested with.

Install Envoy Gateway

Envoy Gateway installs the Gateway API CRDs and controller:

helm install eg \
  oci://docker.io/envoyproxy/gateway-helm \
  --version v1.8.1 \
  --namespace envoy-gateway-system \
  --create-namespace \
  --wait

Create the GatewayClass

Create the eg GatewayClass that the OpenShell chart references:

kubectl apply -f - <<'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: eg
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
EOF

Verify the GatewayClass is accepted:

kubectl get gatewayclass eg

The ACCEPTED column should show True.

Install OpenShell with Gateway API enabled

Enable the GRPCRoute and let the chart create a Gateway resource in the openshell namespace:

helm upgrade --install openshell \
  oci://ghcr.io/nvidia/openshell/helm-chart \
  --version <version> \
  --namespace openshell \
  --set grpcRoute.enabled=true \
  --set grpcRoute.gateway.create=true \
  --set grpcRoute.gateway.className=eg

Get the external address

After the Gateway is provisioned, Envoy Gateway creates a LoadBalancer service in the openshell namespace. Wait for it to get an external address:

kubectl -n openshell get svc -l gateway.envoyproxy.io/owning-gateway-name=openshell

After the EXTERNAL-IP is assigned, register the gateway with the CLI:

openshell gateway add http://<external-ip> --name production
openshell status

This setup is plaintext end-to-end and is intended for development. For external access, terminate TLS at the gateway as shown below.

HTTPS (TLS termination)

Envoy Gateway can terminate TLS at the listener and forward plaintext to the OpenShell gateway pod:

client → HTTPS → Envoy Gateway (terminate TLS) → plaintext → openshell gateway pod

Envoy Gateway only terminates TLS here — it does not perform OIDC. Do not enable an Envoy Gateway OIDC SecurityPolicy in front of the gateway: that flow relies on browser redirects and cookies and cannot work with the OpenShell CLI or headless agents. Instead, the OpenShell gateway validates an OIDC bearer token that the client sends in the gRPC authorization metadata, which Envoy forwards untouched.

Because Envoy terminates TLS, the OpenShell gateway never sees a client certificate, so client mTLS cannot provide identity on this path. Use OIDC bearer tokens for client identity instead.

For an interactive login on a headless machine, set OPENSHELL_NO_BROWSER=1 before running openshell gateway add. The CLI uses the Device Authorization Grant with S256 PKCE and prompts the user to approve the login from another browser. For unattended agents and CI, set OPENSHELL_OIDC_CLIENT_SECRET to use the OAuth2 client-credentials grant instead. The client id comes from --oidc-client-id (default openshell-cli); pass it explicitly if your identity provider uses a different id. Interactive users with a local browser get the Authorization Code flow with PKCE by default.

Provide a TLS certificate

Create a kubernetes.io/tls Secret in the openshell namespace with the certificate for your external hostname:

kubectl -n openshell create secret tls openshell-ingress-tls \
  --cert=tls.crt --key=tls.key

The Secret may also be issued by cert-manager, or you can reference the chart’s existing openshell-server-tls Secret if its SANs include the external hostname.

Install with HTTPS termination

Enable an HTTPS listener, point it at the Secret, disable gateway-pod TLS so Envoy forwards plaintext, and configure an OIDC issuer for client identity:

helm upgrade --install openshell \
  oci://ghcr.io/nvidia/openshell/helm-chart \
  --version <version> \
  --namespace openshell \
  --set grpcRoute.enabled=true \
  --set grpcRoute.gateway.create=true \
  --set grpcRoute.gateway.className=eg \
  --set grpcRoute.gateway.listener.protocol=HTTPS \
  --set grpcRoute.gateway.listener.port=443 \
  --set 'grpcRoute.gateway.listener.tls.certificateRefs[0].name=openshell-ingress-tls' \
  --set server.disableTls=true \
  --set server.oidc.issuer=https://keycloak.example.com/realms/openshell \
  --set server.oidc.audience=openshell-cli \
  --set 'grpcRoute.hostnames[0]=gateway.example.com'

Keep the certificate Secret in the release namespace. Referencing a Secret in another namespace requires a ReferenceGrant.

Register over HTTPS

openshell gateway add https://gateway.example.com \
  --name production \
  --oidc-issuer https://keycloak.example.com/realms/openshell \
  --oidc-client-id openshell-cli
openshell status

See Authentication for OIDC issuer, audience, and roles configuration.

End-to-end TLS (BackendTLSPolicy)

As an alternative to the plaintext backend path above, the chart can create a BackendTLSPolicy that tells the Gateway proxy to re-encrypt traffic when connecting to the OpenShell gateway pod:

client → HTTPS → Gateway (terminate TLS) → TLS (re-encrypt) → openshell gateway pod

This keeps TLS on the gateway pod rather than disabling it with server.disableTls=true. The Gateway proxy validates the backend’s certificate against a CA ConfigMap that the certgen hook auto-creates.

BackendTLSPolicy is a standard Gateway API resource. It is supported on OpenShift 4.22+ (via the OpenShift gateway controller) and on other platforms where the Gateway API implementation supports it (check your controller’s documentation).

Install with e2e TLS

The certgen hook automatically creates the backend CA ConfigMap when backendTLSPolicy is enabled:

helm upgrade --install openshell \
  oci://ghcr.io/nvidia/openshell/helm-chart \
  --version <version> \
  --namespace openshell \
  --set server.tls.enableMtls=false \
  --set grpcRoute.enabled=true \
  --set grpcRoute.gateway.create=true \
  --set grpcRoute.gateway.className=eg \
  --set grpcRoute.gateway.listener.protocol=HTTPS \
  --set grpcRoute.gateway.listener.port=443 \
  --set 'grpcRoute.gateway.listener.tls.certificateRefs[0].name=openshell-ingress-tls' \
  --set grpcRoute.backendTLSPolicy.enabled=true \
  --set server.oidc.issuer=https://keycloak.example.com/realms/openshell \
  --set server.oidc.audience=openshell-cli \
  --set 'grpcRoute.hostnames[0]=gateway.example.com'

Note that server.disableTls is not set — the gateway pod continues to serve TLS — but server.tls.enableMtls=false disables mTLS client certificate authentication because the Gateway proxy cannot present a client certificate to the backend. The chart will fail the install if you try to enable both grpcRoute.backendTLSPolicy.enabled=true and server.tls.enableMtls=true simultaneously. The BackendTLSPolicy hostname defaults to the service FQDN, which matches the SAN on the server certificate. Use OIDC for authentication (configured via server.oidc.issuer).

The example above uses the default pkiInitJob for TLS, which creates the backend CA ConfigMap immediately. If using cert-manager instead (--set certManager.enabled=true), the Certificate resources are regular release objects, and a separate post-install/post-upgrade Job (<release>-certgen-backend-ca) polls for up to 120 seconds waiting for cert-manager to issue the server certificate, then creates the backend CA ConfigMap. This means a single helm install is sufficient in most cases.

If cert-manager takes longer than 120 seconds to issue certificates, increase the polling timeout with --set pkiInitJob.timeoutSeconds=<seconds>. The hook polls for exactly this many seconds. For example, timeoutSeconds=180 polls for 180 seconds. By default (pkiInitJob.failOnTimeout=true), the install fails if the timeout is reached, providing clear feedback that the BackendTLSPolicy is non-functional.

Troubleshooting

If you see the error remote connection failure, transport failure reason: TLS error: Secret is not supplied by SDS when connecting through the Gateway:

  1. Check if the backend CA ConfigMap exists:

    kubectl get configmap <release-name>-backend-ca -n <namespace>

  2. If the ConfigMap is missing, verify the TLS secret exists:

    kubectl get secret <release-name>-server-tls -n <namespace>

  3. If the secret exists but the ConfigMap doesn’t, run helm upgrade to create it:

    helm upgrade <release-name> oci://ghcr.io/nvidia/openshell/helm-chart \
      --reuse-values --namespace <namespace>

This situation can occur if you set pkiInitJob.failOnTimeout=false and cert-manager issued the certificate after the hook timed out.

For OpenShift 4.22+, see OpenShift for platform-specific instructions including Gateway and GatewayClass setup.

SSH Relay

Sandbox SSH uses the gateway endpoint registered with the CLI. No separate Helm SSH host or port values are required.

Next Steps

Return to Setup to complete the installation.