Skip to navigation

Deployment Secrets

View as Markdown

Create and mount the secret material on this page before you set the platform config that turns the feature on. Platform config stores file paths and environment-variable names. It does not store key material. A missing signing key makes token minting fail, and a missing session or client-secret variable makes server-side login return 503.

Do not put these values in platformConfig, a ConfigMap, the Studio bundle, or discovery output.

What each feature needs

FeatureMaterialWhere the process reads itConfig that turns it on
Workload token exchangeUnencrypted PEM RSA private keyFile at auth.token_signing.private_key_fileauth.oidc.workload is set
Kubernetes workload exchangeTokenReview permission for the API ServiceAccountKubernetes APIauth.oidc.workload.kubernetes_token_review_enabled: true
Public-client web sessions (optional)High-entropy encryption keyEnvironment variable named by auth.oidc.server_sessions.encryption_key_env_varauth.oidc.public_client.server_side_sessions: true
Confidential clientThe session encryption key, plus the IdP client secretTwo environment variablesauth.oidc.confidential_client is set

A confidential client always uses server-side sessions, so it always needs the session encryption key. A public client uses them only when you set public_client.server_side_sessions: true. When both are on, they read the same server_sessions key. Token exchange does not use that key. It uses the RSA signing key. Scoped Access Keys use the same RSA key unless auth.oidc.workload.token_private_key_file points at a different file. When those two files differ and Scoped Access Keys are enabled, auth.oidc.workload.token_key_id must differ from auth.token_signing.key_id.

Token signing key

Generate an unencrypted RSA private key. The auth service loads it with no passphrase and accepts only an RSA key.

openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out private-key.pem

Store private-key.pem in your secret manager, sync it into a Kubernetes Secret, and mount that Secret read-only on the API process. For example:

api:
extraVolumes:
- name: token-signing
secret:
secretName: nemo-token-signing
defaultMode: 0440
items:
- key: private-key.pem
path: private-key.pem
extraVolumeMounts:
- name: token-signing
mountPath: /etc/nhx/workload-token
readOnly: true

Point token signing at that file, and use an issuer clients can reach:

auth:
token_signing:
issuer: "https://nhx.company.com/apis/auth"
key_id: "nemo-helix-signing"
private_key_file: "/etc/nhx/workload-token/private-key.pem"
oidc:
enabled: true
workload:
client_id: "nemo-helix-workload"
token_endpoint: "https://nhx.company.com/apis/auth/token"
audience: "nemo-helix"
scope: "openid email groups"
kubernetes_token_review_enabled: true

Set workload.token_endpoint to this deployment’s https://<public-host>/apis/auth/token. Keep a backup of the private key before you enable exchange or Scoped Access Keys. Replacing the key invalidates tokens already signed with that key id.

After the API rolls, confirm the mount:

kubectl -n <namespace> exec deploy/<api-deployment> -- test -r /etc/nhx/workload-token/private-key.pem

Kubernetes TokenReview

With kubernetes_token_review_enabled: true, the API ServiceAccount must be allowed to create tokenreviews.authentication.k8s.io. The chart does not create this role. Apply it before the flag, and substitute the API ServiceAccount name and namespace from the live deployment:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: nemo-helix-token-reviewer
rules:
- apiGroups: ["authentication.k8s.io"]
resources: ["tokenreviews"]
verbs: ["create"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: nemo-helix-token-reviewer
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: nemo-helix-token-reviewer
subjects:
- kind: ServiceAccount
name: nemo-helix-api
namespace: nemo-helix

TokenReview is a cluster-scoped API, so the role is a ClusterRole. The identity that applies it needs permission to create that ClusterRole and bind it. Without the binding, job pods cannot exchange their projected service-account tokens.

After rollout, discovery should report the workload profile, and GET /apis/auth/jwks should list token_signing.key_id.

Session encryption key

Server-side sessions encrypt provider refresh tokens with this key. A confidential client requires it. A public client needs it only when public_client.server_side_sessions is true. Generate the key with:

openssl rand -base64 32

Use that value. Helix hashes it with SHA-256 and uses the digest as a Fernet key, so you do not format a Fernet key yourself. A password or other short string can be guessed from stored refresh tokens. Inject the generated value as the environment variable named by server_sessions.encryption_key_env_var. The usual name is NHX_AUTH_SESSION_ENCRYPTION_KEY.

api:
env:
NHX_AUTH_SESSION_ENCRYPTION_KEY:
valueFrom:
secretKeyRef:
name: nemo-oidc-client
key: session-encryption-key
auth:
oidc:
server_sessions:
encryption_key_env_var: NHX_AUTH_SESSION_ENCRYPTION_KEY
public_client:
client_id: "<public-client-id>"
server_side_sessions: true

Use one value for the whole deployment. The confidential client and public-client web sessions read the same server_sessions key. Replacing the value makes stored refresh tokens undecryptable, and users must log in again.

Studio does not use the login broker. Leave public_client.server_side_sessions false on a deployment that serves Studio.

Confidential client secret

Register the confidential client at the identity provider first. See Confidential OIDC Client. Then store the IdP client secret in a second key and inject it as the variable named by confidential_client.client_secret_env_var. The usual name is NHX_OIDC_CLIENT_SECRET.

The confidential client also requires the session encryption key above. Mount both environment variables before you add auth.oidc.confidential_client.

Order

  1. Create the key material in the secret manager.
  2. Sync it into a Kubernetes Secret and mount the file or inject the environment variables.
  3. Apply the TokenReview binding when Kubernetes workload exchange will be enabled.
  4. Set the platform config that enables the feature, and roll the API.
  5. Confirm the file or variables are present on the API pods.
  6. Check GET /apis/auth/discovery, then a real CLI login or a job that calls the API.