Secure Deployment Guidelines

NVIDIA Dynamo is a high-performance distributed inference platform designed to be deployed within a secure, trusted environment. A deployment coordinates many parts — a frontend, backend workers, a KV router, and internal communication planes for discovery, events, and requests — across a cluster. This guide describes Dynamo’s trust model and how to secure each part of a deployment.

Dynamo is designed to be deployed within a trusted network boundary: external clients reach only the frontend, through an authenticating gateway, and the internal communication planes and infrastructure run on a network isolated from untrusted access. The sections below explain that boundary and provide details on how to further harden different aspects, including further hardening of the internal communication planes. Users deploying Dynamo should make their own decision in balancing functionality, performance, and security within the trusted network boundary.

Throughout this guide, deployer refers to the team responsible for deploying and operating Dynamo in a cluster — distinct from the Dynamo Operator, the Kubernetes operator component.

Trust Model

Dynamo separates client-facing traffic from internal coordination:

• The external-facing inference API — the frontend’s OpenAI-compatible HTTP endpoint, which serves inference requests to clients.
• The internal communication planes — discovery, event, and request planes, plus infrastructure services (NATS, ModelExpress, and the NIXL/RDMA data-transfer fabric) that components use to find each other and coordinate.

The security posture rests on two assumptions:

1. The internal communication planes and infrastructure services are deployed by the deployer in a secure fashion and reside within a trusted network that external clients cannot reach.
2. External clients reach only the frontend, and only through a gateway or proxy that terminates authentication and TLS.

If both hold, the externally reachable surface is limited to the frontend’s inference API. The sections below explain how to satisfy each assumption.

The gateway is the only entry point into the cluster for external clients; it forwards to the frontends (the external-facing inference API). Everything inside — the frontends, the internal communication planes, and the workers — runs on the trusted network and must be protected from access outside the cluster.

Securing the External-Facing Inference API

Do not expose the Dynamo frontend directly to an untrusted network. Deploy it as a microservice behind a dedicated gateway or proxy that provides:

• Authentication and authorization of clients.
• TLS termination and encryption in transit.
• Rate limiting and request-size limits.
• Load balancing across frontend replicas.

On Kubernetes, place a standard ingress or Gateway that you configure for authentication and TLS in front of the Dynamo Frontend service. The frontend implements no client authentication; that is the gateway’s responsibility. If you adopt Dynamo’s optional Gateway API routing topology, note that its Endpoint Picker selects a backend for load and KV-cache reasons and does not authenticate clients, so it still sits behind your authenticating gateway.

TLS termination at the gateway secures only the client-to-gateway hop. If traffic from the gateway to the frontend crosses an untrusted segment, re-encrypt that hop by enabling the frontend’s own server-side TLS (DYN_TLS_CERT_PATH / DYN_TLS_KEY_PATH); see the frontend TLS configuration. This is server-side TLS only — it does not authenticate end users, which remains the gateway’s responsibility.

Securing the Internal Communication Planes

Dynamo components coordinate over three internal communication planes — discovery, event, and request. All three are intended to run within the trusted network and must never be reachable by untrusted clients. Secure each as follows.

Discovery plane

Workers register their endpoints and are discovered through the discovery plane.

• Recommended — Kubernetes-based discovery. Set DYN_DISCOVERY_BACKEND=kubernetes. Dynamo discovers workers through RBAC-gated custom resources; reads and writes are authorized by the Kubernetes API server using each pod’s ServiceAccount, so there is no anonymous, network-reachable discovery store to protect.
• etcd (legacy — local, bare-metal, or non-Kubernetes only). etcd discovery is deprecated for Kubernetes; the Dynamo Operator uses Kubernetes-native discovery by default, and a Kubernetes deployment should not add a second network-reachable discovery store. Where you do use etcd, enable authentication — never anonymous access on a shared network. Dynamo’s etcd client supports username/password (ETCD_AUTH_USERNAME/ETCD_AUTH_PASSWORD) and mutual TLS (ETCD_AUTH_CA, ETCD_AUTH_CLIENT_CERT, ETCD_AUTH_CLIENT_KEY).

Why it matters: an unauthenticated discovery plane lets any peer on the network enumerate workers and inject or alter routing metadata. See the Discovery Plane reference.

Event plane

Components exchange coordination events — including the KV-cache events used for KV-aware routing — over the event plane. Dynamo supports two event transports, selected by DYN_EVENT_PLANE:

• NATS supports authentication — NATS_AUTH_USERNAME/NATS_AUTH_PASSWORD, or NATS_AUTH_TOKEN, NATS_AUTH_NKEY, or NATS_AUTH_CREDENTIALS_FILE — and TLS (NATS_TLS_CA_CERT_PATH). Whether the NATS server requires these is enforced by the NATS server configuration (for example tls { ca_file: …; verify: true }), not by Dynamo, so harden the server config as well.
• ZMQ is the default event transport in configurations without NATS, and some backends (for example, vLLM) publish KV-cache events over ZMQ natively. Dynamo’s ZMQ transports do not add authentication or encryption. Treat all KV-cache events as sensitive request-derived data. Raw engine-side stored-block events can contain token IDs and cache/LoRA context; the token IDs can expose block-aligned request text when decoded with the corresponding tokenizer. Dynamo’s normalized event-plane payloads instead carry deterministic per-token-block hashes, which can reveal equality or shared-prefix relationships and support offline dictionary attacks against predictable token blocks. Neither representation necessarily exposes the complete prompt. Keep every ZMQ endpoint on the trusted network: bind the broker (ZMQ_BROKER_XSUB_BIND / ZMQ_BROKER_XPUB_BIND) and the advertised KV-event host to cluster-internal addresses, keep intra-node sockets on loopback, and restrict who can publish or subscribe with NetworkPolicy.

Why it matters: the event plane carries sensitive request-derived data — KV events can disclose or help infer prompt content, depending on the publisher and wire representation — so keep it on the trusted network and restrict publishers and subscribers. See the Event Plane reference.

Request plane

Requests and KV-cache data move between components over the request plane. The transport is selected by DYN_REQUEST_PLANE — tcp (with the NIXL/RDMA fabric for data transfer) or nats — and the TLS/mTLS settings below apply to whichever you use.

• Encrypt the request plane with TLS. For the TCP transport, enable TLS with DYN_TCP_TLS_CERT_PATH + DYN_TCP_TLS_KEY_PATH (server side); clients verify the server with DYN_TCP_TLS_CA_CERT_PATH (and DYN_TCP_TLS_SERVER_NAME to pin the expected name). For the NATS transport, enable TLS with NATS_TLS_CA_CERT_PATH. This protects the confidentiality and integrity of request-plane traffic in transit.
• Authenticate both ends with mutual TLS (mTLS). Set DYN_TCP_TLS_CLIENT_CA_CERT_PATH on the server so it requires clients to present a certificate — an unauthenticated client is then rejected at the handshake; clients present an identity with DYN_TCP_TLS_CLIENT_CERT_PATH / DYN_TCP_TLS_CLIENT_KEY_PATH. For the NATS transport, use NATS_TLS_CLIENT_CERT_PATH / NATS_TLS_CLIENT_KEY_PATH.
• Constrain the trust domain. TLS clients must validate the server certificate against an appropriately constrained trust root and the expected server identity. mTLS servers must likewise require client certificates from a trust domain scoped to the intended callers. Certificate authentication does not replace authorization or network isolation.
• Keep the NIXL/RDMA data-transfer fabric on the trusted network.

Why it matters: TLS encrypts request-plane traffic, and mTLS additionally authenticates the client so an unauthenticated peer cannot deliver requests or data-transfer payloads to a worker. Keep the plane on the trusted network as defense in depth. See request-plane TLS and mTLS.

Restrict or Disable Optional Surfaces

Dynamo exposes optional control and extension surfaces beyond plain inference. Disable the ones you do not need so that only the required capabilities are reachable.

Frontend extensions and admin API

• Client-controlled routing (nvext). By default the frontend honors an nvext request extension and routing-override headers that let a client pin a request to a specific worker instance. In a multi-tenant or untrusted-client setting, the deployer may want to set DYN_DISABLE_FRONTEND_NVEXT=1 so clients cannot target individual workers. This drops request.nvext at handler entry and ignores the routing-override headers.
• Admin API. The frontend’s HTTP admin API (for example, GET/POST /busy_threshold) is enabled by default. If the deployer does not need to change runtime tunables through it, set DYN_DISABLE_FRONTEND_ADMIN_API=1. Inference, metrics, models, health, and liveness routes are unaffected.
• Metrics endpoint. The /metrics endpoint is intended for scraping by trusted monitoring systems; scope it to your observability stack rather than exposing it to untrusted networks.

Internal control and diagnostic interfaces

Dynamo’s internal control and diagnostic interfaces — for example the worker system server (/engine/* on DYN_SYSTEM_PORT), the planner live dashboard and plugin registration, and the standalone KV router services (indexer, selection, and slot tracker) — are designed to be used within the trusted network boundary. While some provide their own authentication, many do not, and several bind on all interfaces.

Keep every such interface on the trusted network, expose only the routes a deployment uses, and disable the ones you don’t need. In operator-managed deployments, the Dynamo Operator enables the worker system server on DYN_SYSTEM_PORT=9090 and exposes it through an in-cluster ClusterIP Service (the standalone runtime default is -1, disabled). Network reachability is not authorization — that ClusterIP Service is reachable cluster-wide with no caller authentication. Deployers should restrict these interfaces to authorized workloads and namespaces (for example with a NetworkPolicy) and verify that isolation. Where an interface offers its own authentication, enable it as defense in depth — but treat it as additional hardening within the boundary, not a replacement for network isolation. See each component’s documentation for its specific listeners and controls.

Securing Model and Backend Code

Dynamo loads models, tokenizers, chat templates, and (depending on the backend) executable model code. Treat all of these as code that runs with the worker’s privileges.

• Load models only from trusted sources. Restrict which model repositories and registries workers may pull from, and restrict write access to any shared model cache or storage so that only trusted principals can publish artifacts.
• Be deliberate about remote code execution options. Framework options that execute model-supplied Python (for example, trust_remote_code) should be enabled only for models you trust. A pre-existing trust_remote_code / --trust-remote-code flag in a deployment template or worker config does not by itself indicate that a deployer reviewed and approved it — some stock templates ship it — so audit templates and pin the provenance of any such flag.
• Pin what you run. Pin container images by digest and models to an immutable commit/revision (or a reviewed local snapshot), and review generated deployment configuration rather than trusting a templated flag to reflect your intent.
• Validate request-derived values. When integrating or extending Dynamo, validate values taken from a request before using them in security-sensitive operations such as outbound network requests, file paths, or deserialization.

Running with Least Privilege

Kubernetes deployments

• Use RBAC and grant each component’s ServiceAccount only the permissions it needs. The Dynamo Operator ships with scoped roles; do not broaden them.
• Apply NetworkPolicies so the internal communication planes, the workers, and the NIXL/RDMA fabric are reachable only by the components that need them, and never from outside the cluster boundary. The operator does not ship a NetworkPolicy by default.
• Run pods as non-root with a read-only root filesystem where possible, drop unneeded Linux capabilities, and set resource limits.

Standalone Docker deployments

Apply standard Docker hardening: restrict container network access to the trusted segment, set CPU/memory limits, avoid --privileged, drop unneeded capabilities, and run as a non-root user.

Reporting a Vulnerability

To report a potential security vulnerability in Dynamo, follow the process in SECURITY.md — NVIDIA PSIRT via the Security Vulnerability Submission Form or psirt@nvidia.com. Do not open a public GitHub issue for security reports.