Database TLS#

Overview#

The DPS server connects to PostgreSQL using the pgx driver, which supports TLS modes from plaintext (disable) through full certificate verification (verify-full) and PostgreSQL client-certificate authentication (mutual TLS).

Enable database TLS when any of the following apply:

  • PostgreSQL uses a certificate issued by a private or internal CA that is not in the pod’s system trust roots.

  • Your operations policy requires sslmode=verify-full (hostname verification).

  • PostgreSQL is configured with cert authentication and requires a client certificate and private key instead of (or in addition to) a password.

This guide covers:

  • Installation — which Helm values and environment variables control database TLS.

  • Enablement — step-by-step to turn on server-cert verification and/or mutual TLS.

  • Troubleshooting — certificate rotation, handshake-failure diagnostics, and the dps_db_tls_handshake_failures_total metric.

Installation#

Database TLS is configured under global.postgresql in values.yaml and by a matching set of environment variables or CLI flags on the dps-server binary.

Helm values#

Values key

Purpose

Default

global.postgresql.sslMode

Postgres sslmode — one of disable, allow, prefer, require, verify-ca, verify-full

"" (effective default disable)

global.postgresql.sslRootCert

Path inside the pod to a PEM CA certificate used to verify the Postgres server

"" (feature off; uses system roots)

global.postgresql.sslCert

Path inside the pod to a PEM client certificate (mutual TLS)

""

global.postgresql.sslKey

Path inside the pod to a PEM client private key (mutual TLS; pair with sslCert)

""

All three certificate and key fields hold filesystem paths, not certificate contents. Mount the material through dps.secrets. Leave global.postgresql.sslMode unset, or set it to disable, only for deployments whose approved network design does not use PostgreSQL TLS.

For 0.9 overlays, move database.sslMode, database.sslRootCert, database.sslCert, and database.sslKey to the corresponding global.postgresql.* keys. The chart reads the nested keys.

Environment variables#

Variable

Maps to

Notes

DB_SSLMODE

global.postgresql.sslMode

PostgreSQL sslmode

DB_TLS_CA_FILE

global.postgresql.sslRootCert

Path to the CA certificate

DB_TLS_CERT_FILE

global.postgresql.sslCert

Path to the client certificate

DB_TLS_KEY_FILE

global.postgresql.sslKey

Path to the client private key

CLI flags#

dps-server \
  --db-ssl-mode="verify-full" \
  --db-tls-ca-file="/secrets/postgres/ca.crt" \
  --db-tls-cert-file="/secrets/postgres/client.crt" \
  --db-tls-key-file="/secrets/postgres/client.key"

Enablement#

The following walk-through enables sslmode=verify-full with both a CA certificate and mutual-TLS client credentials. Adjust the set of files you mount depending on whether you need server-cert verification only, client auth only, or both.

1. Create the Kubernetes Secret#

Place your CA cert, client cert, and client key into a single Opaque Secret:

kubectl create secret generic dps-postgres-tls \
  --from-file=ca.crt=/path/to/ca.crt \
  --from-file=client.crt=/path/to/client.crt \
  --from-file=client.key=/path/to/client.key \
  -n dps

If you only need server-cert verification (no client auth), omit client.crt and client.key.

2. Set the Helm values#

In your values.yaml overlay:

global:
  postgresql:
    sslMode: "verify-full"
    sslRootCert: "/secrets/postgres/ca.crt"
    sslCert: "/secrets/postgres/client.crt"
    sslKey: "/secrets/postgres/client.key"

dps:
  secrets:
    - name: dps-postgres-tls
      secretKey: ca.crt
      mountPath: /secrets/postgres/ca.crt
    - name: dps-postgres-tls
      secretKey: client.crt
      mountPath: /secrets/postgres/client.crt
    - name: dps-postgres-tls
      secretKey: client.key  # pragma: allowlist secret
      mountPath: /secrets/postgres/client.key  # pragma: allowlist secret

The paths under global.postgresql must match the mountPath entries in dps.secrets. The Helm template emits the corresponding DB_TLS_* certificate environment variables into the dps-server pod only when the values are non-empty. If global.postgresql.sslMode is left unset, the chart defaults to disable. Use verify-ca or verify-full for server-certificate verification.

3. Apply the chart#

helm upgrade --install dps ngc/dps \
  --namespace dps \
  --version "<release-version>" \
  --values values.yaml \
  --wait

4. Verify the handshake#

After the dps-server pod becomes ready, confirm that:

  • The pod log contains a postgres connecting entry with sslmode=verify-full and mtls_enabled=true (neither the DSN nor the password is ever logged).

  • The dps_db_tls_handshake_failures_total counter is 0 (scrape /metrics or query Prometheus directly).

  • Server health endpoints respond normally (no repeated restart loop).

If the counter is non-zero or the pod fails to start, refer to Troubleshooting.

Troubleshooting#

Certificate rotation#

Certificate rotation is not automatic. After rotating a certificate file on disk, the dps-server pod must be restarted to pick up the new certificate (the server loads X.509 key pairs once at startup). Integrate the restart with your approved certificate-rotation procedure.

Operationally, this means:

kubectl rollout restart statefulset/dps-server -n dps

after any rotation of the ca.crt / client.crt / client.key contents inside the Secret.

Handshake-failure metric#

dps_db_tls_handshake_failures_total is a counter with a bounded reason label. It increments once per failed TLS handshake during pool initialization:

reason label

Meaning

Typical cause

cert_parse

CA, client cert, or client key could not be parsed as PEM

Wrong file format, truncated secret, or CRLF injected into PEM

cert_verify

Server cert chain does not chain to the configured CA

Wrong CA, server cert issued by a different CA, or server cert expired

hostname_mismatch

sslmode=verify-full and the server cert SAN does not match the connection host

The configured Postgres host (global.postgresql.host / DB_HOST) does not match the cert SAN — use a hostname that matches the server certificate

protocol

TLS negotiation failed at protocol level

TLS version mismatch, cipher mismatch, or Postgres not configured for TLS

auth_rejected

TLS handshake succeeded but Postgres rejected the client cert / role

Client cert CN does not match a Postgres role, or pg_hba.conf entry requires password

Query example:

sum by (reason) (rate(dps_db_tls_handshake_failures_total[5m]))

Non-zero rate on any label is an operational issue. cert_parse is almost always a Secret-contents problem; cert_verify / hostname_mismatch are CA or server-cert issues; auth_rejected points at Postgres server-side pg_hba.conf.

Common failures#

  • Pod CrashLoopBackOff with “open /secrets/postgres/…: no such file” — a path under global.postgresql is a typo or does not match any mountPath in dps.secrets. Correct the path and reapply.

  • Pod CrashLoopBackOff with “permission denied” — the mounted key file mode, owner, or pod security context does not let the DPS process read it. For the default non-root DPS pod, prefer a group-readable Secret projection such as 0440 with dps.securityContext.fsGroup set to the reader group. Use 0400 only when the file owner is the container UID or the file is otherwise guaranteed readable by that UID.

  • sslcert supplied but no sslkey — pgx rejects this at ParseConfig. Supply both or neither.