Keycloak (air-gapped)#

Keycloak is the identity provider that authenticates users through OpenID Connect.

In an air-gapped environment, install Keycloak and its dependencies from the Helm charts in the bundle artifacts. Registry mirroring serves the container images from your local registry. Install NMC Gateway first (NMC Gateway (air-gapped)). Keycloak is reached through it.

Install the Keycloak components in this order:

  1. PostgreSQL, through CloudNativePG, the database backend for Keycloak.

  2. Keycloak, the identity and access management server.

Then configure the realm that holds the NMC users, either through the Keycloak Admin Console or with keycloak-config-cli.

Note

The components in this section are third-party open-source projects. The configurations here are recommendations. Adjust them to suit your environment and security requirements.

BCM LDAP authentication (optional)#

BCM provides an LDAP directory of its users. Keycloak can federate that directory into a realm over a mutual TLS connection, letting those users sign in through Keycloak. To set this up, follow Prepare BCM LDAP mutual TLS during the Keycloak install and Federate BCM LDAP during Keycloak configuration.

Install PostgreSQL#

Keycloak requires a PostgreSQL database. If you already have one, skip this section and set the database connection in the Keycloak values file to your existing details. The following steps use CloudNativePG to deploy a PostgreSQL cluster.

The PostgreSQL volumes use the cluster’s default StorageClass and require the shared storage verified during Kubernetes installation.

Install the CloudNativePG operator

Create a values file for the operator:

cat <<'EOF' > values-cnpg-operator.yaml
resources:
  limits:
    cpu: 100m
    memory: 200Mi
  requests:
    cpu: 100m
    memory: 100Mi

monitoring:
  podMonitorEnabled: false
  grafanaDashboard:
    create: false
EOF

Set podMonitorEnabled and grafanaDashboard.create to true if Prometheus Operator and Grafana are installed and you want monitoring.

Install the operator from the bundle:

helm install cnpg ./bundle/helm/cloudnative-pg-0.28.2.tgz \
  --namespace keycloak --create-namespace -f values-cnpg-operator.yaml \
  --wait --timeout 5m

--wait holds until the operator is ready. The operator validates the PostgreSQL cluster through an admission webhook, so the next step is rejected while the operator is still starting.

Deploy a PostgreSQL cluster

Create a values file for the cluster:

cat <<'EOF' > values-cnpg-cluster.yaml
cluster:
  instances: 2
  storage:
    size: 2Gi

  monitoring:
    enabled: false

  initdb:
    database: keycloak
    secret:
      name: ""
EOF

Key parameters:

  • cluster.instances: number of PostgreSQL instances. Set it to 2 or higher for high availability.

  • cluster.storage.size: storage size for each instance.

  • cluster.initdb.database: name of the database to create. It must match database.database in values-keycloakx.yaml, which is keycloak in this guide.

  • cluster.initdb.secret.name: name of an existing Kubernetes secret with database credentials. Leave it empty to let CloudNativePG generate them.

Install the cluster from the bundle:

helm install keycloak-cnpg ./bundle/helm/cluster-0.6.1.tgz \
  --namespace keycloak -f values-cnpg-cluster.yaml

Wait for the PostgreSQL cluster to become ready:

kubectl wait --for=condition=Ready cluster/keycloak-cnpg-cluster \
  -n keycloak --timeout=10m

Retrieve the auto-generated database password for the Keycloak configuration:

kubectl get secret keycloak-cnpg-cluster-app -n keycloak \
  -o jsonpath='{.data.password}' | base64 -d

Record the password. The Keycloak values file in the next section needs it.

The PostgreSQL service is reachable in the cluster at host keycloak-cnpg-cluster-rw.keycloak.svc.cluster.local, port 5432, database keycloak, username keycloak.

Install Keycloak#

Prepare BCM LDAP mutual TLS (optional)

To federate BCM LDAP, Keycloak needs a client certificate signed by the BCM certificate authority (refer to BCM LDAP authentication). Prepare the certificate before you write the Keycloak values file.

On the BCM head node, sign a client certificate with the BCM certificate authority and package it as a PKCS#12 keystore. Set an export password when prompted:

openssl genrsa -out client.key 2048
openssl req -new -key client.key -out client.csr -subj "/CN=keycloak-ldap-client"
openssl x509 -req -in client.csr -sha256 -days 365 \
  -CA /cm/local/apps/openldap/etc/certs/ca.pem \
  -CAkey /cm/local/apps/openldap/etc/certs/ca.key -CAcreateserial \
  -out client.crt
openssl pkcs12 -export -in client.crt -inkey client.key \
  -name ldap-client -out ldap-keystore.p12

Create a secret, replacing <keystore-password> with the export password you set:

kubectl create secret generic keycloak-ldap-tls -n keycloak \
  --from-file=ldap-keystore.p12=ldap-keystore.p12 \
  --from-file=ldap-ca.pem=/cm/local/apps/openldap/etc/certs/ca.pem \
  --from-literal=keystore-password=<keystore-password>

Create the values file

The values file needs these values:

  • <your-admin-password>: password for the Keycloak administrator account. Change it from the default for production.

  • <your-keycloak-domain>: the fully qualified domain name for Keycloak, for example keycloak.nmc.example.com. Resolve it as described in Reach the NMC interfaces, and make sure the gateway certificate covers that name.

  • <your-database-password>: the PostgreSQL password you recorded in the previous section.

  • <bcm-ldap-ip>: the BCM LDAP server address. The BCM LDAP values file uses this one only.

To find the BCM LDAP server address, run the following command on the BCM head node:

# Replace <bcm-ldap-ip> in the values file with this output.
getent hosts ldapserver.cm.cluster | awk '{print $1; exit}'

Select the tab that matches your setup. Each tab holds a complete values file. Replace the placeholders in it, then run the command.

cat <<'EOF' > values-keycloakx.yaml
image:
  repository: quay.io/keycloak/keycloak
  tag: "26.6.2"

command:
  - /bin/bash
  - -c
  - |
    export JAVA_OPTS_APPEND="-Djavax.net.ssl.keyStore=/tls/ldap-keystore.p12 \
      -Djavax.net.ssl.keyStorePassword=${KC_LDAP_KEYSTORE_PASSWORD} \
      -Djavax.net.ssl.keyStoreType=PKCS12"
    exec /opt/keycloak/bin/kc.sh start

extraEnv: |
  - name: KEYCLOAK_ADMIN
    value: admin
  - name: KEYCLOAK_ADMIN_PASSWORD
    value: "<your-admin-password>"
  - name: KC_HTTP_MANAGEMENT_ENABLED
    value: "true"
  - name: KC_HTTP_MANAGEMENT_RELATIVE_PATH
    value: "/"
  - name: KC_HOSTNAME
    value: "https://<your-keycloak-domain>"
  - name: KC_HOSTNAME_STRICT
    value: "false"
  - name: KC_HOSTNAME_STRICT_HTTPS
    value: "true"
  - name: KC_HOSTNAME_STRICT_BACKCHANNEL
    value: "false"
  - name: KC_HOSTNAME_BACKCHANNEL_DYNAMIC
    value: "true"
  - name: KC_TRUSTSTORE_PATHS
    value: /tls/ldap-ca.pem
  - name: KC_LDAP_KEYSTORE_PASSWORD
    valueFrom:
      secretKeyRef:
        name: keycloak-ldap-tls
        key: keystore-password

extraVolumes: |
  - name: ldap-tls
    secret:
      secretName: keycloak-ldap-tls

extraVolumeMounts: |
  - name: ldap-tls
    mountPath: /tls
    readOnly: true

hostAliases:
  - ip: <bcm-ldap-ip>
    hostnames:
      - ldapserver.cm.cluster

resources:
  limits:
    cpu: 2
    memory: 1024Mi
  requests:
    cpu: 200m
    memory: 512Mi

serviceMonitor:
  enabled: false

metrics:
  enabled: true

health:
  enabled: true

http:
  relativePath: "/"
  internalPort: http-internal
  internalScheme: HTTP

proxy:
  mode: xforwarded

ingress:
  enabled: false

httpRoute:
  enabled: true
  servicePort: 80
  parentRefs:
    - name: nmc-gateway
      namespace: nmc-gateway
      sectionName: https
  hostnames:
    - "<your-keycloak-domain>"

database:
  vendor: postgres
  hostname: keycloak-cnpg-cluster-rw.keycloak.svc.cluster.local
  port: 5432
  database: keycloak
  username: keycloak
  password: "<your-database-password>"
EOF

cat <<'EOF' > values-keycloakx.yaml
image:
  repository: quay.io/keycloak/keycloak
  tag: "26.6.2"

command:
  - "/opt/keycloak/bin/kc.sh"
  - "start"

extraEnv: |
  - name: KEYCLOAK_ADMIN
    value: admin
  - name: KEYCLOAK_ADMIN_PASSWORD
    value: "<your-admin-password>"
  - name: KC_HTTP_MANAGEMENT_ENABLED
    value: "true"
  - name: KC_HTTP_MANAGEMENT_RELATIVE_PATH
    value: "/"
  - name: KC_HOSTNAME
    value: "https://<your-keycloak-domain>"
  - name: KC_HOSTNAME_STRICT
    value: "false"
  - name: KC_HOSTNAME_STRICT_HTTPS
    value: "true"
  - name: KC_HOSTNAME_STRICT_BACKCHANNEL
    value: "false"
  - name: KC_HOSTNAME_BACKCHANNEL_DYNAMIC
    value: "true"

resources:
  limits:
    cpu: 2
    memory: 1024Mi
  requests:
    cpu: 200m
    memory: 512Mi

serviceMonitor:
  enabled: false

metrics:
  enabled: true

health:
  enabled: true

http:
  relativePath: "/"
  internalPort: http-internal
  internalScheme: HTTP

proxy:
  mode: xforwarded

ingress:
  enabled: false

httpRoute:
  enabled: true
  servicePort: 80
  parentRefs:
    - name: nmc-gateway
      namespace: nmc-gateway
      sectionName: https
  hostnames:
    - "<your-keycloak-domain>"

database:
  vendor: postgres
  hostname: keycloak-cnpg-cluster-rw.keycloak.svc.cluster.local
  port: 5432
  database: keycloak
  username: keycloak
  password: "<your-database-password>"
EOF

The httpRoute section attaches Keycloak to NMC Gateway. The gateway terminates TLS and forwards the request to Keycloak over HTTP, which is why proxy.mode is xforwarded.

Install Keycloak from the bundle:

helm install keycloakx ./bundle/helm/keycloakx-7.2.0.tgz \
  --namespace keycloak -f values-keycloakx.yaml

Wait for Keycloak to become ready:

kubectl rollout status statefulset/keycloakx -n keycloak --timeout=5m

Open the Keycloak Admin Console at https://<your-keycloak-domain>/ and sign in with the administrator credentials from the values file.

This step exercises NMC Gateway for the first time. If the console does not open, check these in order:

  • From the machine running the browser, <your-keycloak-domain> resolves as described in Reach the NMC interfaces.

  • The gateway certificate covers <your-keycloak-domain>. A browser refuses a certificate that omits the name.

  • The route attached to the gateway. Run kubectl describe httproute -n keycloak and read the Status section. It lists nmc-gateway as the parent, with Accepted and ResolvedRefs both True. Accepted reports whether the gateway took the route, and ResolvedRefs whether the route found the Keycloak service.

Configure Keycloak#

After Keycloak is running, configure its realms, clients, and identity providers, either programmatically with keycloak-config-cli or manually through the Keycloak Admin Console.

Create a realm for NMC before you configure anything else. NMC Launchpad signs its users in against this realm, and the examples in this guide name it nmc. In the Admin Console, open the realm selector and select Create realm.

Create a user for NMC Launchpad

The administrator account in values-keycloakx.yaml belongs to the master realm and cannot sign in to NMC Launchpad. NMC Launchpad authenticates against the nmc realm, so that realm needs its own users.

Federating BCM LDAP lets the BCM accounts sign in to this realm. Without LDAP federation, create a user through the Admin Console:

  1. Select the nmc realm in the realm selector.

  2. Select Users > Add user.

  3. Enter a Username and an Email, then select Create. Keycloak requires an email address to complete a sign-in, and stops a user without one at the Update Account Information screen.

  4. On the Credentials tab, select Set password, enter the password, and set Temporary to Off.

Federate BCM LDAP (optional)

To federate BCM LDAP (refer to BCM LDAP authentication), configure a user federation provider with the following settings, then apply them using Option 1 or Option 2.

Both options require the bind password for the read-only account. Read it from BCM:

# Use this value for LDAP_BIND_CREDENTIAL.
awk -F'"' '/LDAPReadOnlyPass/{print $2}' /cm/local/apps/cmd/etc/cmd.conf

Set Use Truststore SPI to never so Keycloak uses the Java keystore and presents the client certificate.

Setting

Value

Connection URL

ldaps://ldapserver.cm.cluster:636

Bind DN

cn=readonlyroot,dc=cm,dc=cluster

Bind credential

The LDAPReadOnlyPass value from BCM

Users DN

dc=cm,dc=cluster

Edit mode

READ_ONLY

Use Truststore SPI

never

Import users

On

Periodic changed users sync

On, every 300 seconds

Periodic full sync

On, every 604800 seconds

Option 1: keycloak-config-cli

keycloak-config-cli applies a realm configuration from a file, which suits repeatable, version-controlled setups. The following values file defines the BCM LDAP provider in the nmc realm. Replace nmc with your realm name, and <bind-password> with the LDAPReadOnlyPass value from BCM:

cat <<'EOF' > values-keycloak-config-cli.yaml
image:
  repository: quay.io/adorsys/keycloak-config-cli
  tag: "6.4.1-26.1.0"

env:
  KEYCLOAK_URL: "http://keycloakx-http.keycloak.svc.cluster.local/"
  KEYCLOAK_USER: "admin"
  KEYCLOAK_AVAILABILITYCHECK_ENABLED: "true"
  KEYCLOAK_AVAILABILITYCHECK_TIMEOUT: "120s"
  IMPORT_FILES_LOCATIONS: "/config/*"
  IMPORT_VARSUBSTITUTION_ENABLED: "true"

secrets:
  KEYCLOAK_PASSWORD: "<your-admin-password>" # Password for the Keycloak admin user
  LDAP_BIND_CREDENTIAL: "<bind-password>"

config:
  nmc:
    inline:
      realm: "nmc"
      enabled: true
      components:
        org.keycloak.storage.UserStorageProvider:
          - name: "bcm-ldap"
            providerId: "ldap"
            config:
              enabled: ["true"]
              editMode: ["READ_ONLY"]
              vendor: ["other"]
              connectionUrl: ["ldaps://ldapserver.cm.cluster:636"]
              bindDn: ["cn=readonlyroot,dc=cm,dc=cluster"]
              bindCredential: ["$(env:LDAP_BIND_CREDENTIAL)"]
              usersDn: ["dc=cm,dc=cluster"]
              usernameLDAPAttribute: ["uid"]
              rdnLDAPAttribute: ["uid"]
              uuidLDAPAttribute: ["entryUUID"]
              userObjectClasses: ["inetOrgPerson, posixAccount"]
              customUserSearchFilter: ["(uid=*)"]
              searchScope: ["2"]
              useTruststoreSpi: ["never"]
              pagination: ["true"]
              batchSizeForSync: ["1000"]
              fullSyncPeriod: ["604800"]
              changedSyncPeriod: ["300"]
              importEnabled: ["true"]
            subComponents:
              org.keycloak.storage.ldap.mappers.LDAPStorageMapper:
                - name: "username"
                  providerId: "user-attribute-ldap-mapper"
                  config:
                    ldap.attribute: ["uid"]
                    user.model.attribute: ["username"]
                    read.only: ["true"]
                - name: "first name"
                  providerId: "user-attribute-ldap-mapper"
                  config:
                    ldap.attribute: ["cn"]
                    user.model.attribute: ["firstName"]
                    read.only: ["true"]
                - name: "last name"
                  providerId: "user-attribute-ldap-mapper"
                  config:
                    ldap.attribute: ["sn"]
                    user.model.attribute: ["lastName"]
                    read.only: ["true"]

resources:
  limits:
    cpu: 500m
    memory: 512Mi
  requests:
    cpu: 100m
    memory: 256Mi
EOF

The KEYCLOAK_URL host keycloakx-http comes from the Keycloak release name (keycloakx). Update it if you used a different release name.

Note

These LDAP provider fields follow the Keycloak realm configuration format, which can change between Keycloak versions. For the full set of fields, attribute mappers, and sync options, refer to the latest Keycloak Server Administration Guide.

helm install keycloak-config-cli ./bundle/helm/keycloak-config-cli-1.3.7.tgz \
  --namespace keycloak -f values-keycloak-config-cli.yaml

If the provider already exists, run the same command with helm upgrade --install in place of helm install. Keycloak updates the provider in place and restarts its sync schedule.

Option 2: Keycloak Admin Console

Add the provider under User federation. Refer to the Keycloak Server Administration Guide.

Import the BCM users

Keycloak imports an account into the realm when someone searches for it under Users, when the user first signs in, or when a scheduled sync runs. The first scheduled sync runs about five minutes after you create the provider and imports every account that matches the search filter. Later syncs import the accounts that BCM added or changed since the previous sync. Restarting Keycloak restarts the five-minute wait.

To import the accounts immediately, or to apply a widened search filter or a new Users DN:

  1. Select the nmc realm, then select User federation > bcm-ldap.

  2. Select Action > Sync all users.

  3. Select Users and search for *.

Read the Keycloak log to confirm the account counts:

kubectl logs -n keycloak -l app.kubernetes.io/name=keycloakx --since=10m \
  | grep -E 'Sync (all|changed) users'

A successful sync reports the accounts it imported:

Sync all users finished: 1 imported users, 0 updated users

A sync that finds the accounts already in the realm counts them as updated, so a repeat run reports 0 imported users, 1 updated users.

If the sync imports nothing and a search for * returns nothing, read the Keycloak log for LDAP, TLS, and bind errors, and then confirm the Users DN, the search filter, and the object classes on your BCM accounts.

Verify#

  • You can sign in to the Keycloak Admin Console at your Keycloak host name.

  • If you federated BCM LDAP, searching for * under Users returns the BCM accounts.