TLS

Dynamo supports opt-in TLS encryption on the network transports between its components: the TCP request and response streams between frontends and workers, and connections to NATS. On the TCP transport, both the request plane (frontend → worker inference requests) and the response stream (worker → frontend inference output) are encrypted using rustls with the ring cryptographic provider. When no TLS configuration is provided, these transports operate in plaintext exactly as before.

The DYN_TCP_TLS_* environment variables encrypt the frontend↔worker TCP request and response streams; NATS traffic is encrypted separately via NATS TLS (see the NATS TLS section below). The KV event plane is a separate transport: when it runs over ZMQ it is not encrypted, and when it runs over NATS it is encrypted only if NATS TLS is configured.

Environment variables

All TLS configuration is driven by environment variables. The Rust runtime reads these directly at first connection (lazy initialization).

Both frontends and workers act as TCP server and client depending on the stream direction (response streams: worker dials frontend; request streams: frontend dials worker). All TLS env vars should be set on every pod.

Server role (accepting connections)

Client role (dialing connections)

CLI flags

The same configuration is available via command-line flags on all backends (vllm, sglang, trtllm, tokenspeed) through DynamoRuntimeArgGroup:

--tcp-tls-cert-path PATH      Server certificate (PEM)
--tcp-tls-key-path PATH       Server private key (PEM)
--tcp-tls-ca-cert-path PATH   CA certificate for server verification (PEM)
--tcp-tls-insecure             Disable certificate verification
--tcp-tls-server-name NAME     Override TLS SNI hostname
--tcp-tls-handshake-timeout N  Handshake timeout in seconds (default: 3)

The frontend (dynamo.frontend) also accepts --tcp-tls-cert-path, --tcp-tls-key-path, and --tcp-tls-ca-cert-path.

Quick start

Generate a self-signed certificate for local testing:

# Generate CA
openssl req -x509 -newkey rsa:2048 -keyout ca-key.pem -out ca-cert.pem \
  -days 365 -nodes -subj "/CN=DynamoCA"
# Generate server cert with SAN
openssl req -newkey rsa:2048 -keyout server-key.pem -out server-csr.pem \
  -nodes -subj "/CN=localhost" \
  -addext "subjectAltName=DNS:localhost,IP:127.0.0.1"
openssl x509 -req -in server-csr.pem -CA ca-cert.pem -CAkey ca-key.pem \
  -CAcreateserial -out server-cert.pem -days 365 -copy_extensions copyall

Both frontend and worker need the same flags (both act as server and client):

python -m dynamo.vllm \
  --tcp-tls-cert-path server-cert.pem \
  --tcp-tls-key-path server-key.pem \
  --tcp-tls-ca-cert-path ca-cert.pem \
  --tcp-tls-server-name localhost \
  ...
python -m dynamo.frontend \
  --tcp-tls-cert-path server-cert.pem \
  --tcp-tls-key-path server-key.pem \
  --tcp-tls-ca-cert-path ca-cert.pem \
  --tcp-tls-server-name localhost \
  ...

Kubernetes deployment

In Kubernetes, TLS certificates are typically delivered by a certificate management system and mounted into pods. Set the environment variables on each component’s pod template in the DynamoGraphDeployment spec:

spec:
  components:
  - name: Frontend
    podTemplate:
      spec:
        containers:
        - name: main
          env:
          - name: DYN_TCP_TLS_CERT_PATH
            value: /etc/certs/server/cert.pem
          - name: DYN_TCP_TLS_KEY_PATH
            value: /etc/certs/server/key.pem
          - name: DYN_TCP_TLS_CA_CERT_PATH
            value: /etc/certs/ca/ca.pem
  - name: VllmWorker
    podTemplate:
      spec:
        containers:
        - name: main
          env:
          - name: DYN_TCP_TLS_CERT_PATH
            value: /etc/certs/server/cert.pem
          - name: DYN_TCP_TLS_KEY_PATH
            value: /etc/certs/server/key.pem
          - name: DYN_TCP_TLS_CA_CERT_PATH
            value: /etc/certs/ca/ca.pem

Both components need the same TLS env vars because each acts as both TCP server and client depending on the stream direction.

For platform-wide TLS that the operator injects into every deployment automatically — instead of setting env vars on each component — see Operator TLS.

NATS TLS

NATS carries JetStream indexer recovery/replay, the audit sink, and — when the request plane is set to NATS (--request-plane nats) — inference request distribution. TLS is configured separately from TCP:

When only the tls:// URL scheme is used without explicit TLS env vars, async-nats handles TLS natively with system roots.

The NATS_SERVER URL accepts both nats:// and tls:// schemes, but setting any explicit NATS TLS variable (NATS_TLS_*) requires the tls:// scheme — startup fails with a clear error otherwise.

CLI flags (all backends via DynamoRuntimeArgGroup):

--nats-tls-ca-cert-path PATH   CA certificate for NATS server verification (PEM)
--nats-tls-insecure             Disable NATS certificate verification

The frontend (dynamo.frontend) also accepts --nats-tls-ca-cert-path and --nats-tls-insecure.

Enabling TLS on the NATS server

The env vars above configure the client side (Dynamo verifying the NATS server). For TLS to work, the NATS server itself must also be configured to listen on TLS — otherwise a client that sets NATS_TLS_CA_CERT_PATH (or the operator-level natsTLSCAPath) will attempt a TLS handshake against a plaintext port and fail.

When deploying Dynamo’s platform chart, the bundled NATS subchart exposes this via nats.config.nats.tls. The certificates are typically delivered by a certificate management system such as cert-manager.

One-way TLS (server presents a cert, clients verify it):

# NATS server side — platform chart values
nats:
  config:
    nats:
      tls:
        enabled: true
        secretName: nats-server-tls   # Secret with tls.crt / tls.key (e.g. cert-manager)

Or, if the cert files are mounted into the pod by an external system, use the merge block to point at the on-disk paths directly:

nats:
  config:
    nats:
      tls:
        enabled: true
        merge:
          cert_file: /etc/certs/server/cert.pem
          key_file:  /etc/certs/server/key.pem
          timeout:   2

Then point the Dynamo clients at the TLS endpoint and give them the CA. These are operator-subchart values, so nest them under dynamo-operator: when using the platform chart:

dynamo-operator:
  natsAddr: "tls://dynamo-platform-nats.dynamo-system.svc.cluster.local:4222"
  natsTLSCAPath: /etc/certs/ca/ca.pem   # → NATS_TLS_CA_CERT_PATH

mTLS (server also verifies client certs) — add ca_file and verify to the server’s merge block, and present a client identity from the Dynamo side:

# NATS server side
nats:
  config:
    nats:
      tls:
        enabled: true
        merge:
          cert_file: /etc/certs/server/cert.pem
          key_file:  /etc/certs/server/key.pem
          ca_file:   /etc/certs/client-ca/ca.pem   # verify client certs
          verify:    true                          # require client certs

# Dynamo client side (operator-level, nested under dynamo-operator:)
dynamo-operator:
  natsTLSCAPath:         /etc/certs/ca/ca.pem
  natsTLSClientCertPath: /etc/certs/client/cert.pem
  natsTLSClientKeyPath:  /etc/certs/client/key.pem

See the NATS TLS documentation for the full server-side TLS schema.

Mutual TLS (mTLS)

By default only the server is authenticated (the client verifies the server’s certificate). Mutual TLS additionally makes the client present a certificate that the server verifies, so both ends of a connection prove their identity. mTLS is opt-in and layered on top of the TLS configuration above.

For the TCP transports (request plane + response stream) Dynamo owns both ends, so configure two things and — since every component is both TCP client and server — set both on every pod:

• On the client: a client certificate/key to present.
• On the server: a CA certificate (DYN_TCP_TLS_CLIENT_CA_CERT_PATH) to verify the presented client certificate. When set, the server requires a trusted client certificate and rejects the handshake otherwise.

For NATS the broker is external, so Dynamo only presents a client identity — there is no Dynamo-side “server” half. Whether client certificates are actually required is enforced by the NATS server configuration (e.g. tls { ca_file: ...; verify: true }), not by Dynamo.

TCP request/response streams

NATS

Note: Dynamo cannot detect whether the NATS broker actually requests a client certificate. If you set NATS_TLS_CLIENT_CERT_PATH/_KEY_PATH but the broker is not configured to verify client certs, the connection silently falls back to ordinary one-way TLS. Enable verification on the NATS server (verify: true) to make NATS mTLS effective.

Client certificate and key must be set together, and a client identity requires a server CA (*_CA_CERT_PATH) — or insecure mode for development — so the server can still be verified. The TCP servers and the NATS client validate this at startup and fail closed. The TCP client connectors, however, initialize lazily on the first outbound connection, so an incomplete TCP client config surfaces as a per-connection handshake failure rather than a startup crash. The client certificate/key hot-reload from disk on rotation, exactly like the server certificate (see Design notes).

The same options are available as CLI flags on all backends (via DynamoRuntimeArgGroup) and the frontend (dynamo.frontend):

--tcp-tls-client-cert-path PATH      Client certificate presented for TCP mTLS (PEM)
--tcp-tls-client-key-path PATH       Client private key for TCP mTLS (PEM)
--tcp-tls-client-ca-cert-path PATH   CA to verify client certs; enforces TCP mTLS (PEM)
--nats-tls-client-cert-path PATH     Client certificate presented for NATS mTLS (PEM)
--nats-tls-client-key-path PATH      Client private key for NATS mTLS (PEM)

Encrypted paths

When TLS is configured, the following transports are encrypted (the ZMQ event plane is not covered):

Design notes

• Server certificates hot-reload: the request-plane and response-stream servers serve their leaf cert/key through a resolver that re-reads the files from disk when their contents change (detected by a content hash, so rotations done by an atomic symlink swap are handled too), so certificate rotation takes effect without a process restart. The check is rate-limited (at most once every 30s, sooner after a failed reload) and never blocks a handshake; a failed reload keeps serving the last valid certificate. Client trust anchors (the CA) are still loaded once, so rotating the CA itself requires a restart.
• mTLS client identity certificates hot-reload through the same resolver, so a rotated client cert/key is picked up without a restart. The CA used by the server to verify client certificates is loaded once (rotating it needs a restart).
• Client TLS connectors are built once and cached via OnceCell on the first outbound connection.
• The TLS handshake is spawned per-connection on both the request plane and response stream servers so the accept loop is never blocked.
• Invalid TLS configuration on the request plane (e.g. bad cert path) prevents server startup rather than silently falling back to plaintext.
• When server and client TLS configurations are mismatched (e.g., server has TLS but client does not), a warning is logged at startup.
• An empty CA certificate file is detected at load time and rejected with a clear error message.