TCP TLS

Dynamo supports opt-in TLS encryption on the TCP request and response streams between frontends and workers. When enabled, both the request plane (frontend → worker inference requests) and the response stream (worker → frontend inference output) are encrypted using rustls with the ring cryptographic provider. When no TLS configuration is provided, these streams operate in plaintext exactly as before.

The same DYN_TCP_TLS_* environment variables apply to both transports — a single set of certificates encrypts the frontend↔worker TCP request and response streams. This does not cover the KV event plane, which is carried over ZMQ or NATS Core depending on setup and is a separate transport; those paths are not encrypted by this configuration.

Environment variables

All TLS configuration is driven by environment variables. The Rust runtime reads these directly at first connection (lazy initialization).

Both frontends and workers act as TCP server and client depending on the stream direction (response streams: worker dials frontend; request streams: frontend dials worker). All TLS env vars should be set on every pod.

Server role (accepting connections)

Client role (dialing connections)

CLI flags

The same configuration is available via command-line flags on all backends (vllm, sglang, trtllm, tokenspeed) through DynamoRuntimeArgGroup:

--tcp-tls-cert-path PATH      Server certificate (PEM)
--tcp-tls-key-path PATH       Server private key (PEM)
--tcp-tls-ca-cert-path PATH   CA certificate for server verification (PEM)
--tcp-tls-insecure             Disable certificate verification
--tcp-tls-server-name NAME     Override TLS SNI hostname
--tcp-tls-handshake-timeout N  Handshake timeout in seconds (default: 3)

The frontend (dynamo.frontend) also accepts --tcp-tls-cert-path, --tcp-tls-key-path, and --tcp-tls-ca-cert-path.

Quick start

Generate a self-signed certificate for local testing:

$
# Generate CA
$
openssl req -x509 -newkey rsa:2048 -keyout ca-key.pem -out ca-cert.pem \
>
  -days 365 -nodes -subj "/CN=DynamoCA"
$
$
# Generate server cert with SAN
$
openssl req -newkey rsa:2048 -keyout server-key.pem -out server-csr.pem \
>
  -nodes -subj "/CN=localhost" \
>
  -addext "subjectAltName=DNS:localhost,IP:127.0.0.1"
$
$
openssl x509 -req -in server-csr.pem -CA ca-cert.pem -CAkey ca-key.pem \
>
  -CAcreateserial -out server-cert.pem -days 365 -copy_extensions copyall

Both frontend and worker need the same flags (both act as server and client):

$
python -m dynamo.vllm \
>
  --tcp-tls-cert-path server-cert.pem \
>
  --tcp-tls-key-path server-key.pem \
>
  --tcp-tls-ca-cert-path ca-cert.pem \
>
  --tcp-tls-server-name localhost \
>
  ...
$
$
python -m dynamo.frontend \
>
  --tcp-tls-cert-path server-cert.pem \
>
  --tcp-tls-key-path server-key.pem \
>
  --tcp-tls-ca-cert-path ca-cert.pem \
>
  --tcp-tls-server-name localhost \
>
  ...

Kubernetes deployment

In Kubernetes, TLS certificates are typically delivered by a certificate management system and mounted into pods. Set the environment variables on each component’s pod template in the DynamoGraphDeployment spec:

1
spec:
2
  components:
3
  - name: Frontend
4
    podTemplate:
5
      spec:
6
        containers:
7
        - name: main
8
          env:
9
          - name: DYN_TCP_TLS_CERT_PATH
10
            value: /etc/certs/server/cert.pem
11
          - name: DYN_TCP_TLS_KEY_PATH
12
            value: /etc/certs/server/key.pem
13
          - name: DYN_TCP_TLS_CA_CERT_PATH
14
            value: /etc/certs/ca/ca.pem
15
  - name: VllmWorker
16
    podTemplate:
17
      spec:
18
        containers:
19
        - name: main
20
          env:
21
          - name: DYN_TCP_TLS_CERT_PATH
22
            value: /etc/certs/server/cert.pem
23
          - name: DYN_TCP_TLS_KEY_PATH
24
            value: /etc/certs/server/key.pem
25
          - name: DYN_TCP_TLS_CA_CERT_PATH
26
            value: /etc/certs/ca/ca.pem

Both components need the same TLS env vars because each acts as both TCP server and client depending on the stream direction.

Note: A future PR (#10809) will add operator-level TLS configuration via InfrastructureConfiguration, allowing TLS to be configured once at the platform level and auto-injected into all DGD pods without per-component env var setup.

Encrypted paths

When TLS is configured, the following frontend↔worker streams are encrypted:

Design notes

• Server certificates hot-reload: the request-plane and response-stream servers serve their leaf cert/key through a resolver that re-reads the files from disk when their contents change (detected by a content hash, so rotations done by an atomic symlink swap are handled too), so certificate rotation takes effect without a process restart. The check is rate-limited (at most once every 30s, sooner after a failed reload) and never blocks a handshake; a failed reload keeps serving the last valid certificate. Client trust anchors (the CA) are still loaded once, so rotating the CA itself requires a restart.
• Client TLS connectors are built once and cached via OnceCell on the first outbound connection.
• The TLS handshake is spawned per-connection on both the request plane and response stream servers so the accept loop is never blocked.
• Invalid TLS configuration on the request plane (e.g. bad cert path) prevents server startup rather than silently falling back to plaintext.
• When server and client TLS configurations are mismatched (e.g., server has TLS but client does not), a warning is logged at startup.
• An empty CA certificate file is detected at load time and rejected with a clear error message.