> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# TCP TLS

Dynamo supports opt-in TLS encryption on the TCP request and response streams
between frontends and workers. When enabled, both the **request plane**
(frontend → worker inference requests) and the **response stream** (worker →
frontend inference output) are encrypted using
[rustls](https://github.com/rustls/rustls) with the `ring` cryptographic
provider. When no TLS configuration is provided, these streams operate in
plaintext exactly as before.

The same `DYN_TCP_TLS_*` environment variables apply to both transports — a
single set of certificates encrypts the frontend↔worker TCP request and
response streams. This does **not** cover the KV event plane, which is carried
over ZMQ or NATS Core depending on setup and is a separate transport; those
paths are not encrypted by this configuration.

## Environment variables

All TLS configuration is driven by environment variables. The Rust runtime
reads these directly at first connection (lazy initialization).

Both frontends and workers act as TCP server and client depending on the
stream direction (response streams: worker dials frontend; request streams:
frontend dials worker). All TLS env vars should be set on every pod.

### Server role (accepting connections)

| Variable                | Description                                                                                                        |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------ |
| `DYN_TCP_TLS_CERT_PATH` | Path to the PEM certificate file. When set together with `DYN_TCP_TLS_KEY_PATH`, TLS is enabled on the TCP server. |
| `DYN_TCP_TLS_KEY_PATH`  | Path to the PEM private key for the server certificate.                                                            |

### Client role (dialing connections)

| Variable                             | Description                                                                                              |
| ------------------------------------ | -------------------------------------------------------------------------------------------------------- |
| `DYN_TCP_TLS_CA_CERT_PATH`           | Path to the PEM CA certificate used to verify the peer's server certificate.                             |
| `DYN_TCP_TLS_INSECURE`               | Set to `1` or `true` to skip certificate verification. For local development only.                       |
| `DYN_TCP_TLS_SERVER_NAME`            | Override the TLS SNI hostname. Useful when connecting by IP to a server whose certificate has a DNS SAN. |
| `DYN_TCP_TLS_HANDSHAKE_TIMEOUT_SECS` | TLS handshake timeout in seconds (default: 3).                                                           |

## CLI flags

The same configuration is available via command-line flags on all backends
(vllm, sglang, trtllm, tokenspeed) through `DynamoRuntimeArgGroup`:

```
--tcp-tls-cert-path PATH      Server certificate (PEM)
--tcp-tls-key-path PATH       Server private key (PEM)
--tcp-tls-ca-cert-path PATH   CA certificate for server verification (PEM)
--tcp-tls-insecure             Disable certificate verification
--tcp-tls-server-name NAME     Override TLS SNI hostname
--tcp-tls-handshake-timeout N  Handshake timeout in seconds (default: 3)
```

The frontend (`dynamo.frontend`) also accepts `--tcp-tls-cert-path`,
`--tcp-tls-key-path`, and `--tcp-tls-ca-cert-path`.

## Quick start

Generate a self-signed certificate for local testing:

```bash
# Generate CA
openssl req -x509 -newkey rsa:2048 -keyout ca-key.pem -out ca-cert.pem \
  -days 365 -nodes -subj "/CN=DynamoCA"

# Generate server cert with SAN
openssl req -newkey rsa:2048 -keyout server-key.pem -out server-csr.pem \
  -nodes -subj "/CN=localhost" \
  -addext "subjectAltName=DNS:localhost,IP:127.0.0.1"

openssl x509 -req -in server-csr.pem -CA ca-cert.pem -CAkey ca-key.pem \
  -CAcreateserial -out server-cert.pem -days 365 -copy_extensions copyall
```

Both frontend and worker need the same flags (both act as server and client):

```bash
python -m dynamo.vllm \
  --tcp-tls-cert-path server-cert.pem \
  --tcp-tls-key-path server-key.pem \
  --tcp-tls-ca-cert-path ca-cert.pem \
  --tcp-tls-server-name localhost \
  ...

python -m dynamo.frontend \
  --tcp-tls-cert-path server-cert.pem \
  --tcp-tls-key-path server-key.pem \
  --tcp-tls-ca-cert-path ca-cert.pem \
  --tcp-tls-server-name localhost \
  ...
```

## Kubernetes deployment

In Kubernetes, TLS certificates are typically delivered by a certificate
management system and mounted into pods. Set the
environment variables on each component's pod template in the
`DynamoGraphDeployment` spec:

```yaml
spec:
  components:
  - name: Frontend
    podTemplate:
      spec:
        containers:
        - name: main
          env:
          - name: DYN_TCP_TLS_CERT_PATH
            value: /etc/certs/server/cert.pem
          - name: DYN_TCP_TLS_KEY_PATH
            value: /etc/certs/server/key.pem
          - name: DYN_TCP_TLS_CA_CERT_PATH
            value: /etc/certs/ca/ca.pem
  - name: VllmWorker
    podTemplate:
      spec:
        containers:
        - name: main
          env:
          - name: DYN_TCP_TLS_CERT_PATH
            value: /etc/certs/server/cert.pem
          - name: DYN_TCP_TLS_KEY_PATH
            value: /etc/certs/server/key.pem
          - name: DYN_TCP_TLS_CA_CERT_PATH
            value: /etc/certs/ca/ca.pem
```

Both components need the same TLS env vars because each acts as both TCP
server and client depending on the stream direction.

> **Note:** A future PR ([#10809](https://github.com/ai-dynamo/dynamo/issues/10809))
> will add operator-level TLS configuration via `InfrastructureConfiguration`,
> allowing TLS to be configured once at the platform level and auto-injected
> into all DGD pods without per-component env var setup.

## Encrypted paths

When TLS is configured, the following frontend↔worker streams are encrypted:

| Path            | Direction         | Data                            | Transport                                           |
| --------------- | ----------------- | ------------------------------- | --------------------------------------------------- |
| Request plane   | Frontend → Worker | User prompts, request metadata  | `egress/tcp_client` → `ingress/shared_tcp_endpoint` |
| Response stream | Worker → Frontend | Inference output tokens         | `tcp/client` → `tcp/server`                         |
| Request stream  | Frontend → Worker | Streaming input (bidirectional) | `tcp/client` → `tcp/server`                         |

## Design notes

* Server certificates hot-reload: the request-plane and response-stream servers
  serve their leaf cert/key through a resolver that re-reads the files from disk
  when their contents change (detected by a content hash, so rotations done by
  an atomic symlink swap are handled too), so certificate rotation
  takes effect **without a process restart**. The check is rate-limited (at most
  once every 30s, sooner after a failed reload) and never blocks a handshake; a
  failed reload keeps serving the last valid certificate. Client trust anchors
  (the CA) are still loaded once, so rotating the CA itself requires a restart.
* Client TLS connectors are built once and cached via `OnceCell` on the first
  outbound connection.
* The TLS handshake is spawned per-connection on both the request plane and
  response stream servers so the accept loop is never blocked.
* Invalid TLS configuration on the request plane (e.g. bad cert path) prevents
  server startup rather than silently falling back to plaintext.
* When server and client TLS configurations are mismatched (e.g., server has TLS
  but client does not), a warning is logged at startup.
* An empty CA certificate file is detected at load time and rejected with a
  clear error message.