Pre-Deployment

Everything that must be in place before deploying the DSX Event Bus. This covers infrastructure prerequisites, secrets provisioning, NKey generation, certificate management, and Gateway setup.

Estimated time: The production path (secrets pipeline + certificates) takes 4–6 hours for a first-time deployment across 1 CSC + 2 CPCs. The evaluation path (local/ Makefile) takes ~10 minutes. See Deployment — Evaluation Install for the quick-start option.

Host Prerequisites

A multi-cluster deployment (CSC + CPCs) creates enough kubelets, containerd shims, gateway controllers, and fsnotify watchers to exhaust default Linux inotify limits. Symptoms include too many open files from kubectl logs -f, silent fsnotify watcher failures, and sporadic kubectl exec errors.

Verify these sysctl parameters on each node before creating clusters:

$
sudo sysctl -w fs.inotify.max_user_instances=8192
$
sudo sysctl -w fs.inotify.max_user_watches=524288

To persist across reboots, add to /etc/sysctl.d/ or equivalent for your OS. For Kind-based local evaluation, see local/README.md for additional macOS-specific setup (MetalLB networking).

Infrastructure Prerequisites

The following must be installed in each Kubernetes cluster before deploying the event bus. Components are version-pinned where there is a known API or compatibility break; unpinned components work with any recent release.

Keycloak or another OIDC provider is required if using OAuth2 authentication.

Auth-Callout Container Image

The auth-callout container image is not published to a public registry. Operators must build the image from source and push it to their own container registry before deploying the Helm chart.

$
make -C auth-callout docker-build
$
# Produces auth-callout:latest
$
$
# Tag and push to your registry
$
docker tag auth-callout:latest registry.example.com/dsx-exchange/auth-callout:latest
$
docker push registry.example.com/dsx-exchange/auth-callout:latest

Then set the image in your Helm values:

1
auth-callout:
2
  image:
3
    repository: registry.example.com/dsx-exchange/auth-callout
4
    tag: latest

Required Secrets

All secrets must be provisioned before helm install. Secret names and keys are overridable in Helm values; these are the defaults.

NATS Server Auth

Auth-Callout Service

NACK Controller

Surveyor

Cross-Cluster Leaf Connections

Each CPC gets a nats-leaf-csc secret. The CSC gets the pubkey for each CPC.

mTLS Secrets

The generation script always produces mTLS keys (there is no flag to skip them). These secrets are only consumed when global.eventBus.mtls.enabled: true; they can be ignored for non-mTLS deployments.

Server:

The chart does not create a cert-manager Certificate CR for nats-mtls-server-tls. Operators must provision this secret before deploying. Use cert-manager with your PKI Issuer, or create the secret manually:

$
kubectl -n dsx create secret generic nats-mtls-server-tls \
>
  --from-file=ca.crt=ca.pem \
>
  --from-file=tls.crt=server.pem \
>
  --from-file=tls.key=server-key.pem

The server certificate SANs must include the hostname or IP that mTLS clients will connect to (see Deployment — mTLS Hostname Agreement).

Leaf connections:

When global.eventBus.mtls.enabled: false, none of the mTLS secrets are required and the mTLS NATS cluster is not deployed.

NKey Generation

NKeys are Ed25519 public-key pairs used for NATS authentication. The generation script requires nsc and nk on PATH.

$
nsc generate nkey --user     # user nkey (SU seed, U pubkey)
$
nsc generate nkey --account  # account nkey (SA seed, A pubkey)
$
nsc generate nkey --curve    # xkey (SX seed, X pubkey)

Required Keys

Generation Script

A script generates all required secrets for a cluster. Without CPC IDs, only the CSC output is generated or left unchanged. With CPC IDs, CSC and the requested CPC outputs are generated or left unchanged. For example:

$
./deploy/scripts/generate-nkeys.sh [OPTIONS] [cpc-ids...]
$
$
# Options:
$
#   -o, --output DIR             Output root directory (default: deploy/secrets)
$
#       --extra-account NAME     Generate CPC-to-CSC leaf keys for an extra account
$
#   -h, --help                   Show help message
$
$
# Examples:
$
./deploy/scripts/generate-nkeys.sh                             # CSC only
$
./deploy/scripts/generate-nkeys.sh 1 2 3                       # CSC + CPC-1, CPC-2, CPC-3
$
./deploy/scripts/generate-nkeys.sh --extra-account LaunchLayer 1 2

Each key is written as a subdirectory containing seed and pubkey files:

secrets/{cluster}/
  nkeys/
    nats-auth-signing/seed
    nats-auth-signing/pubkey
    nats-xkey/seed
    nats-xkey/pubkey
    nats-authx-user/seed
    nats-authx-user/pubkey
    nats-nack-user/seed
    nats-nack-user/pubkey
    nats-nack-user/nack-user.nk
    nats-mtls-leaf/seed
    nats-mtls-leaf/pubkey
    nats-mtls-authx-leaf/seed
    nats-mtls-authx-leaf/pubkey
    nats-mtls-sys-leaf/seed
    nats-mtls-sys-leaf/pubkey
    nats-surveyor/seed
    nats-surveyor/pubkey
    auth-callout-keys/nkey-seed
    auth-callout-keys/issuer-seed
    auth-callout-keys/xkey-seed
    nats-leaf-cpc-{id}/pubkey    # CSC only, one per CPC
    xkey.nk

Secrets Pipeline

The event bus consumes Kubernetes Secrets — it does not interact with any secrets backend directly. Any pipeline that materializes the secrets listed in Required Secrets into the target namespace before helm install will work. The Helm chart does not assume Vault, sealed-secrets, external-secrets, or any other specific provider.

One common pattern is OpenBao or HashiCorp Vault with the Vault Secrets Operator (VSO) to materialize secrets and the Vault Agent Injector for auth-callout seed injection. For installation, K8s auth methods, policies, and PKI setup, see the respective documentation:

• OpenBao
• Vault on Kubernetes
• Vault Secrets Operator
• KV Secret Engine
• PKI Secret Engine

Certificate Management

cert-manager handles TLS certificate lifecycle. The Issuer can be backed by any supported provider (Vault/OpenBao PKI, self-signed, ACME, etc.).

Server TLS Certificate

All TLS-terminated Gateway listeners reference a cert-manager Certificate:

1
apiVersion: cert-manager.io/v1
2
kind: Certificate
3
metadata:
4
  name: event-bus-server-tls-certificate
5
spec:
6
  secretName: event-bus-server-tls-certificate
7
  issuerRef:
8
    name: event-bus-certificate-issuer
9
    kind: Issuer
10
  commonName: "event-bus.example.com"
11
  dnsNames:
12
    - "event-bus.example.com"

Gateway Setup

A Gateway API controller must be installed before deploying the event bus. The Gateway resource defines the external listeners that route traffic to the NATS pods.

Gateway Resource

1
apiVersion: gateway.networking.k8s.io/v1
2
kind: Gateway
3
metadata:
4
  name: event-bus-gateway
5
spec:
6
  gatewayClassName: eg
7
  listeners:
8
    - name: mqtt
9
      protocol: TLS
10
      port: 1883
11
      allowedRoutes:
12
        namespaces:
13
          from: All
14
      tls:
15
        mode: Terminate
16
        certificateRefs:
17
          - kind: Secret
18
            name: event-bus-server-tls-certificate
19
    - name: nats-client
20
      protocol: TLS
21
      port: 4222
22
      allowedRoutes:
23
        namespaces:
24
          from: All
25
      tls:
26
        mode: Terminate
27
        certificateRefs:
28
          - kind: Secret
29
            name: event-bus-server-tls-certificate
30
    - name: nats-leafnode
31
      protocol: TLS
32
      port: 7422
33
      allowedRoutes:
34
        namespaces:
35
          from: All
36
      tls:
37
        mode: Terminate
38
        certificateRefs:
39
          - kind: Secret
40
            name: event-bus-server-tls-certificate
41
    - name: mqtt-mtls
42
      protocol: TLS
43
      port: 8883
44
      allowedRoutes:
45
        namespaces:
46
          from: All
47
      tls:
48
        mode: Passthrough
49
  infrastructure:
50
    parametersRef:
51
      group: gateway.envoyproxy.io
52
      kind: EnvoyProxy
53
      name: event-bus-proxy-config

The mqtt-mtls listener uses Passthrough mode because TLS termination happens at the NATS pod to verify the client certificate.

Because the mTLS route is a TLSRoute, the server certificate SANs, the TLSRoute hostnames, and the client broker URL must all agree on the same hostname (or IP). A mismatch causes a silent connection reset at the gateway layer. See Deployment — mTLS Hostname Agreement.

Listener-to-Route Mapping

Gateway listener names must match the sectionName in the Helm gateway.routes values. Route kind depends on the TLS mode:

External References

• Vault KV Secret Engine: https://developer.hashicorp.com/vault/docs/secrets/kv
• Vault PKI Secret Engine: https://developer.hashicorp.com/vault/docs/secrets/pki
• Vault Secrets Operator: https://developer.hashicorp.com/vault/docs/deploy/kubernetes/vso
• cert-manager Vault Issuer: https://cert-manager.io/docs/configuration/vault/
• NATS NKey Auth: https://docs.nats.io/running-a-nats-service/configuration/securing_nats/auth_intro/nkey_auth
• NATS Auth Callout: https://docs.nats.io/running-a-nats-service/configuration/securing_nats/auth_callout