OpenShift

View as Markdown

The Kubernetes driver resolves the UID range assigned to each OpenShift namespace and renders the sandbox and supervisor with a numeric non-root identity from that range. OpenShell does not require the privileged SCC or any added Linux capability.

Verify that the selected OpenShift runtime profile permits an unprivileged process to install a nested seccomp user-notification filter and use Landlock. OpenShell fails sandbox startup when either capability-free runtime probe fails.

Node kernel and legacy read-only mode

OpenShift nodes run RHCOS, which currently ships a RHEL 9.x kernel (5.14). That kernel predates SECCOMP_FILTER_FLAG_WAIT_KILLABLE_RECV (Linux 5.19), so the sandbox starts in a reduced legacy read-only cancellation mode. Isolation is unchanged, but the broker fails closed with EOPNOTSUPP on the mediated operations that write results back into workload memory — getpeername, accept/accept4 with a non-null peer-address argument, and sendmmsg per-message length write-backs. Outbound-oriented workloads run unchanged; server workloads that read the peer address on accept need a node kernel with WAIT_KILLABLE_RECV (Linux 5.19+, or a distribution backport). See the support matrix for the full behavior; the selected mode is reported as seccomp_listener_mode in the sandbox qualification output.

Prerequisites

  • OpenShift 4.x cluster with oc configured.
  • Helm 3.x.
  • Agent Sandbox controller and CRDs.
  • A CNI that enforces ingress and egress NetworkPolicy in sandbox namespaces.

Your cluster MUST enforce ingress and egress NetworkPolicy in every sandbox namespace. OpenShell creates the policies, but Kubernetes does not verify that the CNI applies them. Without enforcement, sandbox workloads may bypass supervisor network policy through direct connections.

Install OpenShell

Pre-create the namespace, then install the chart. Keep the default restricted security posture.

oc create ns openshell
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart \
--version <version> \
--namespace openshell

The driver reads the namespace’s openshift.io/sa.scc.uid-range annotation and uses the resulting UID/GID for the sandbox, agent, trusted init containers, and supervisor. Each container sets allowPrivilegeEscalation: false, drops all Linux capabilities, and uses RuntimeDefault seccomp.

Wait for the gateway:

oc -n openshell rollout status statefulset/openshell

If you set workload.kind=deployment, wait for deployment/openshell instead.

Connect to the Gateway

Forward the gateway port for local evaluation:

oc -n openshell port-forward svc/openshell 8080:8080
openshell gateway add https://127.0.0.1:8080 --local --name openshift
openshell status

Options for end-to-end TLS

The steps above run the gateway over plaintext HTTP for quick evaluation. For production deployments, choose one of the approaches below based on your OpenShift version and preferences.

End-to-end TLS using Gateway API and BackendTLSPolicy (OpenShift 4.22+)

OpenShift 4.22 and later support BackendTLSPolicy in the Gateway API, enabling end-to-end TLS between the OpenShift router and the OpenShell gateway pod. The traffic flow is:

client → HTTPS → OpenShift Gateway (terminate TLS) → TLS (re-encrypt) → openshell gateway pod

This removes the requirement to run the gateway with server.disableTls=true. The OpenShift router terminates client-facing TLS at the listener and re-encrypts when connecting to the backend service, validating the backend’s certificate against a CA you provide.

Prerequisites

  • OpenShift 4.22+ cluster with the Gateway API enabled
  • cert-manager installed (recommended) or the built-in pkiInitJob for server certificates
  • A GatewayClass registered for the OpenShift gateway controller

Create the GatewayClass

If your cluster does not already have an OpenShift GatewayClass, create one:

oc apply -f - <<'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: openshift-default
spec:
controllerName: openshift.io/gateway-controller/v1
EOF

Create the Gateway

Create a Gateway resource in the openshift-ingress namespace. Replace <external-hostname> with your cluster’s route hostname (typically a wildcard like *.openshell-ingress-gw.example.com):

oc apply -f - <<'EOF'
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: openshell-gateway
namespace: openshift-ingress
spec:
gatewayClassName: openshift-default
listeners:
- name: grpc
hostname: "<external-hostname>"
port: 443
protocol: HTTPS
tls:
mode: Terminate
certificateRefs:
- name: <listener-tls-secret>
kind: Secret
allowedRoutes:
namespaces:
from: Selector
selector:
matchLabels:
kubernetes.io/metadata.name: openshell
EOF

The listener TLS Secret should contain the certificate for the external hostname.

Install with e2e TLS

Install the chart with the GRPCRoute and BackendTLSPolicy enabled. The certgen hook automatically creates the backend CA ConfigMap from the generated PKI bundle:

helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart \
--version <version> \
--namespace openshell \
--set podSecurityContext.fsGroup=null \
--set securityContext.runAsUser=null \
--set server.tls.enableMtls=false \
--set grpcRoute.enabled=true \
--set grpcRoute.gateway.name=openshell-gateway \
--set grpcRoute.gateway.namespace=openshift-ingress \
--set 'grpcRoute.hostnames[0]=gateway.example.com' \
--set grpcRoute.backendTLSPolicy.enabled=true \
--set server.oidc.issuer=https://keycloak.example.com/realms/openshell \
--set server.oidc.audience=openshell-cli
OverrideReason
podSecurityContext.fsGroup=null / securityContext.runAsUser=nullLet OpenShift’s SCC admission assign UIDs.
server.tls.enableMtls=falseDisable mTLS client certificate authentication. BackendTLSPolicy only validates the server certificate; the ingress proxy cannot present a client certificate to the backend. Use OIDC for authentication instead.
grpcRoute.enabled=trueCreate a GRPCRoute pointing at the external Gateway.
grpcRoute.gateway.name / namespaceReference the Gateway created above in openshift-ingress.
grpcRoute.backendTLSPolicy.enabled=trueCreate a BackendTLSPolicy for TLS re-encryption to the gateway pod. The certgen hook auto-creates the backend CA ConfigMap. The Gateway proxy validates the backend certificate against the service FQDN, which is already in the default server certificate SANs.
grpcRoute.hostnamesExternal hostname for the GRPCRoute. This goes on the Gateway listener certificate, not the backend certificate.

Note that server.disableTls is not set — the gateway pod serves TLS over HTTPS without requiring client certificates. Use OIDC for authentication (see Access Control).

Using cert-manager instead of pkiInitJob: Add --set certManager.enabled=true to the install command. The default certManager.serverDnsNames already includes the service FQDN needed for BackendTLSPolicy validation. The Certificate resources are regular release objects, and a separate post-install/post-upgrade Job (<release>-certgen-backend-ca) polls for up to 120 seconds waiting for cert-manager to issue the server certificate, then creates the backend CA ConfigMap. A single helm install is sufficient in most cases.

If cert-manager takes longer than 120 seconds to issue certificates, increase the polling timeout with --set pkiInitJob.timeoutSeconds=<seconds>. The hook polls for exactly this many seconds. For example, timeoutSeconds=180 polls for 180 seconds. By default (pkiInitJob.failOnTimeout=true), the install fails if the timeout is reached, providing clear feedback that the BackendTLSPolicy is non-functional.

Register over HTTPS

openshell gateway add https://gateway.example.com \
--name openshift \
--oidc-issuer https://keycloak.example.com/realms/openshell \
--oidc-client-id openshell-cli
openshell status

End-to-end TLS using pass-through Route (all OpenShift versions)

For OpenShift versions prior to 4.22, or when you prefer Route-based ingress, cert-manager can issue the gateway’s server certificate from a real Issuer or ClusterIssuer (for example, a LetsEncrypt/ACME issuer), and an OpenShift Route with TLS passthrough exposes it externally while the gateway keeps terminating its own TLS and mTLS.

Install cert-manager and configure a working ClusterIssuer first — see Managing Certificates for the certManager.serverIssuerRef details. Configure an OIDC provider as described in Access Control — remote gateways authenticate CLI users via OIDC, not mTLS, so the gateway must know the OIDC issuer URL. Install the chart with:

helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart \
--version <version> \
--namespace openshell \
--set podSecurityContext.fsGroup=null \
--set securityContext.runAsUser=null \
--set server.disableTls=false \
--set certManager.enabled=true \
--set certManager.serverIssuerRef.name=letsencrypt-prod \
--set certManager.serverIssuerRef.kind=ClusterIssuer \
--set certManager.serverDnsNames[0]=gateway.example.com \
--set openshiftRoute.enabled=true \
--set openshiftRoute.host=gateway.example.com \
--set server.oidc.issuer=https://keycloak.example.com/realms/openshell \
--set server.oidc.audience=openshell-cli
OverrideReason
certManager.serverIssuerRefCreates a second server certificate from your Issuer or ClusterIssuer for external clients. The gateway uses SNI to present this cert for the external hostname while continuing to present the internal (chart CA) cert to supervisors. The internal certificate’s ca.crt is the chart CA that also signed the client cert, so the default clientCaFromServerTlsSecret=true is correct.
openshiftRoute.enabled / openshiftRoute.hostCreates an OpenShift Route with TLS passthrough — the router forwards the encrypted connection by SNI without decrypting, so the gateway uses the SNI hostname to select the external certificate.
server.oidc.issuer / server.oidc.audienceConfigures server-side OIDC validation. Without these, the gateway expects mTLS client certificates and rejects OIDC-only CLI connections. See Access Control.

Register the gateway with the CLI over OIDC. Remote gateways authenticate CLI users via OIDC, not mTLS — see Access Control:

openshell gateway add https://gateway.example.com \
--name openshift \
--oidc-issuer https://keycloak.example.com/realms/openshell \
--oidc-client-id openshell-cli
openshell gateway login openshift

Next Steps

  • For more on certificate provisioning modes, refer to Managing Certificates.
  • To expose the gateway externally through the Kubernetes Gateway API instead of a Route, refer to Ingress.
  • To configure OIDC authentication, refer to Access Control.