Runtime Configuration (DynamoRuntimeConfig)
Runtime Configuration (DynamoRuntimeConfig)
Field reference for the cross-cutting Dynamo runtime CLI flags and environment variables shared by every backend and the frontend.
DynamoRuntimeConfig holds the configuration that is common to every Dynamo process — the frontend, the standalone router, and all three backend workers (vLLM, SGLang, TRT-LLM) parse this same argument group. Most fields, types, defaults, and choices on this page come from the DynamoRuntimeArgGroup and DynamoRuntimeConfig definitions. The page also catalogs runtime environment-only controls implemented below the Python argument layer.
These are the shared Dynamo runtime flags. Each backend also has its own backend-specific flags: vLLM Configuration, SGLang Configuration, and TensorRT-LLM Configuration. This page covers neither native engine arguments nor backend-specific DYN_* prefixed flags — only the flags from DynamoRuntimeArgGroup.
How the config is loaded
Unless a field is marked environment-only, it has both a CLI flag and an environment variable. The CLI flag takes precedence; the environment variable is the fallback. Boolean fields are negatable — --dyn-enable-structural-tag sets it on, --no-dyn-enable-structural-tag sets it off.
Model discovery and namespace
Dynamo namespace that scopes service discovery. All components in a deployment must share the same namespace to find each other. If DYN_NAMESPACE_WORKER_SUFFIX is also set, -{suffix} is automatically appended to the resolved value to support multiple worker pools serving the same model.
On Kubernetes the operator sets this automatically to {k8s_namespace}-{dgd_name}; override only deliberately.
Environment variable: DYN_NAMESPACE
Dynamo endpoint string in dyn://namespace.component.endpoint format, for example dyn://dynamo.backend.generate. When unset, the endpoint address is inferred from the component’s registration in the discovery backend.
Environment variable: DYN_ENDPOINT
Comma-separated list of OpenAI-compatible endpoint types to enable. Use completions alone for models that do not have a chat template. The obsolete alias --dyn-endpoint-types is accepted for backward compatibility.
Environment variable: DYN_ENDPOINT_TYPES
Communication planes
Service discovery backend. kubernetes uses the K8s API; etcd uses a distributed key-value store (configured via ETCD_* env vars such as ETCD_ENDPOINTS); file uses the local filesystem (path from DYN_FILE_KV, defaulting to $TMPDIR/dynamo_store_kv); mem uses an in-memory store suitable for single-process development. See Discovery Plane for design details.
On Kubernetes the operator sets this automatically to kubernetes; override only deliberately.
Environment variable: DYN_DISCOVERY_BACKEND
Maximum number of seconds that an etcd discovery client spends establishing its initial
connection and primary lease. Dynamo retries with exponential backoff until this deadline. Set a
larger value when etcd can take longer to become available during deployment startup or recovery;
set a smaller value when the process should fail fast. Values must be greater than 0. Zero or
invalid values log a warning and fall back to 120.
Transport used to distribute requests from routers to workers. tcp provides the lowest latency and is recommended for production. nats uses NATS messaging. See Request Plane for design details.
Environment variable: DYN_REQUEST_PLANE
Preferred payload codec advertised by every request-plane endpoint served by this process. Clients
select the codec independently for each destination endpoint. They use the destination’s advertised
codec, or json when a legacy destination does not advertise one. Setting this variable does not
force the codec for outbound requests from the process.
Dynamo caches this process-wide value on its first codec lookup. Restart the process after changing it.
Allowed values: json msgpackMaximum TCP request-plane frame size in bytes (default 32 MiB). The frontend
rejects an oversized frame with HTTP 400 before writing it. This limit is
independent of DYN_HTTP_BODY_LIMIT_MB (default 192 MB) on the HTTP ingress:
a chat-completions body can be accepted by the HTTP server and still fail at
this cap after serialization. Inline data: media is sent once in
multi_modal_data; raise this value when operators must accept larger
inline-media worker frames.
See Frontend Configuration for DYN_HTTP_BODY_LIMIT_MB.
Host for the TCP request-plane server. Accepts IPv4 and IPv6 literals, bracketed IPv6 literals,
wildcards, and interface names, including aliases such as eth0:1. If unset or empty, Dynamo
selects the first usable non-loopback IPv4 address, then IPv6, then IPv4 loopback, then IPv6
loopback. On Unix, automatic selection excludes interfaces that are down. Dynamo trims surrounding
whitespace from configured hosts; a value containing only whitespace still fails server startup.
For a named interface, Dynamo uses the same address selection order, restricted to that interface. It selects the first usable address in each category in operating-system enumeration order. An interface with only loopback addresses uses IPv4 loopback, then IPv6 loopback. Explicit interface lookup does not filter by interface state.
A wildcard uses the same automatic interface inventory to select a usable non-loopback address in its requested family for advertisement. If only the other family has one, Dynamo switches the bind wildcard to that family. If neither family has one, Dynamo prefers loopback in the requested family before trying the other family. If no loopback address is found, it uses the requested family’s standard loopback address. The bind wildcard uses the selected address family. Wildcards are never advertised. IPv4 link-local addresses are accepted. Multicast, broadcast, and IPv6 link-local addresses are rejected.
IPv4-mapped IPv6 addresses use IPv4 rules. For example, ::ffff:0.0.0.0 uses the 0.0.0.0
wildcard rules and advertises a concrete address.
The server resolves the address once at startup. A loopback fallback persists until restart, and an interface-enumeration error fails server startup.
Port used by the TCP request-plane server. Accepted values are integers from 0 to 65535.
When the value is unset, empty, or 0, the operating system assigns a free port. Dynamo does not
trim whitespace. An invalid value, including a value with surrounding whitespace, silently falls
back to an operating-system-assigned port.
Host for the TCP response-stream server. It accepts the same address syntax and uses the same
address selection and wildcard-family rules as DYN_TCP_RPC_HOST. Dynamo trims whitespace and
treats an empty value as unset. With a wildcard host, the TCP listener binds the selected wildcard,
while the QUIC response listener binds only the selected concrete address.
The server resolves the address once at startup. A loopback fallback persists until restart, and an interface-enumeration error fails server startup.
Port used by the TCP response stream server. Dynamo trims surrounding whitespace. When the value
is unset, empty, or 0, the operating system assigns a free port. Values must be between 0 and
65535.
Event publishing transport. ZMQ is the default for every discovery backend; select nats
explicitly to use NATS Core. See Event Plane for design details.
Environment variable: DYN_EVENT_PLANE
IPv4 or IPv6 address, or exact network interface name, advertised by direct ZMQ event publishers.
For an interface name, Dynamo selects the first usable IPv4 address, then IPv6. Dynamo trims
whitespace and treats an empty value as unset. Unspecified addresses such as 0.0.0.0 and :: are
invalid, including IPv4-mapped wildcards such as ::ffff:0.0.0.0. When unset, Dynamo selects a
usable non-loopback IPv4 address first, then IPv6. On Unix, automatic selection excludes down
interfaces. If neither family has a usable non-loopback address, Dynamo selects IPv4 loopback,
then IPv6 loopback, and falls back to 127.0.0.1 if neither is found. Remote subscribers cannot
reach a loopback address. Automatic selection can choose an IPv4 link-local address or a bridge
address. On hosts with several interfaces, set this variable to a reachable IP address or interface
if automatic selection chooses the wrong address.
Direct ZMQ publishers bind 0.0.0.0 for an advertised IPv4 address or [::] for an advertised
IPv6 address. Discovery publishes the concrete address with the port assigned to that listener.
An explicit override must identify an address reachable by subscribers. The IPv6 wildcard can
also accept IPv4 connections; keep ZMQ listeners on trusted networks.
Event-plane PUB/SUB sockets support IPv4 addresses and bracketed IPv6 addresses, including
connections to configured ZMQ brokers. Hostnames retain IPv4 resolution. For an IPv6 broker,
use a bracketed address such as tcp://[2001:db8::10]:5555; Dynamo rejects an unbracketed IPv6
address. This variable does not affect NATS
or ZMQ broker mode.
On IPv6-only hosts, set DYN_HTTP_HOST=:: on the Frontend and DYN_SYSTEM_HOST=:: on workers. Both
listeners default to 0.0.0.0, which accepts IPv4 connections only. The request plane and the direct
ZMQ event plane select an IPv6 address automatically when the host has no usable non-loopback IPv4
address.
Number of libzmq background I/O threads in the process-wide event-plane context. All event-plane PUB/SUB sockets in a frontend or worker process share this context. This setting does not change the Tokio runtime size or the number of KV indexer threads.
Set a positive integer before starting the process. Dynamo reads the value when it first creates
the shared context; invalid values fail context initialization. Restart the process to change it.
Set 1 to restore the previous single-thread setting. This variable does not affect NATS or ZMQ
contexts created outside the shared event-plane transport.
KV-cache transfer connector. Accepts zero or more values. Deprecated for vLLM — use --kv-transfer-config instead. For TRT-LLM, valid options are nixl, lmcache, kvbm, null, and none.
Environment variable: DYN_CONNECTOR
Parsing
Tool call parser name for the model. When unset, the backend’s default tool call parsing is used. Valid parser names are determined at runtime by the installed dynamo._core extension — run python -m dynamo.vllm --help to see the available choices on your installation.
Environment variable: DYN_TOOL_CALL_PARSER
Reasoning/chain-of-thought parser name for the model. When unset, no reasoning parsing is performed. Valid parser names are determined at runtime by the installed dynamo._core extension — run python -m dynamo.vllm --help to see the available choices on your installation.
Environment variable: DYN_REASONING_PARSER
Thinking mode to apply when a request omits an explicit thinking control. Request-level thinking, reasoning_effort, and controls in chat_template_args or chat_template_kwargs take precedence. When unset, Dynamo preserves the model or chat template’s native default. See Deployment-Level Thinking Default for examples and precedence details.
Environment variable: DYN_DEFAULT_THINKING_MODE
Exclude tool definitions from the chat template when tool_choice='none'. Prevents models from generating unsolicited raw XML tool calls in the content field. This flag controls the Rust-native chat template path; a matching flag in FrontendArgGroup controls the Python processor side independently.
Environment variable: DYN_EXCLUDE_TOOLS_WHEN_TOOL_CHOICE_NONE
Enable structural tag guided decoding for tool calls. When enabled, configure activation scope and parameter schema strictness with --dyn-structural-tag-scope and --dyn-structural-tag-schema.
Environment variable: DYN_ENABLE_STRUCTURAL_TAG
Controls when structural tags are activated. auto activates them for required or named tool_choice, or when any tool has strict=true or parallel_tool_calls is false. always additionally activates them for tool_choice=auto without those conditions. tool_choice=none is unaffected by either setting. Only meaningful when --dyn-enable-structural-tag is set.
Environment variable: DYN_STRUCTURAL_TAG_SCOPE
Controls parameter schema strictness inside structural tags. auto applies the real parameter schema only to tools with strict=true, leaving all other tools syntactically constrained but schema-unconstrained. strict applies the real parameter schema to every tool. Only meaningful when --dyn-enable-structural-tag is set.
Environment variable: DYN_STRUCTURAL_TAG_SCHEMA
Path to a custom Jinja template file to override the model’s default chat template. This template takes precedence over any template found in the model repository.
Environment variable: DYN_CUSTOM_JINJA_TEMPLATE
Multimodal and media output
Capacity of the multimodal embedding cache in GB. Set to 0 to disable the cache entirely. Increase this for workloads with repeated image or audio inputs to avoid redundant re-encoding.
Environment variable: DYN_MULTIMODAL_EMBEDDING_CACHE_CAPACITY_GB
Space-separated list of output modalities for omni or diffusion mode, for example --output-modalities text image. Defaults to text only. Use image, video, or audio when deploying image/video/audio generation models.
Environment variable: DYN_OUTPUT_MODALITIES
Filesystem URL for storing generated images and videos. Accepts a local file:// path or a remote object-storage URL such as s3://bucket/path.
Environment variable: DYN_MEDIA_OUTPUT_FS_URL
Base HTTP URL for rewriting media file paths in API responses, for example http://localhost:8000/media. When unset, raw filesystem paths are returned in the response body.
Environment variable: DYN_MEDIA_OUTPUT_HTTP_URL
Fault tolerance
Maximum number of seconds that a runtime client locally removes a worker from normal routing after
a request-path failure while service discovery propagates the worker state. Set to 0 to disable
local inhibition. The process reads this value once when the first client initializes; restart the
process after changing it. Discovery updates remain authoritative, and direct dispatch bypasses the
local inhibited set while the selected worker remains in discovery.
Operations
Dump the fully resolved configuration to the specified file path at startup. Useful for auditing the effective values after all env-var and CLI overrides have been applied.
Environment variable: DYN_DUMP_CONFIG_TO
Override the runtime health-check canary payload used by the unified backend’s Worker. Accepts a JSON object string, for example '{"token_ids": [1], "stop_conditions": {"max_tokens": 1}}', or a file reference prefixed with @, for example @/path/to/payload.json. Takes precedence over the engine’s default health_check_payload(). Applies to the unified backend only.
Environment variable: DYN_HEALTH_CHECK_PAYLOAD
Maximum requests handled concurrently by the engine. Setting a positive integer enables worker-side admission control. When the engine slots and the Dynamo overflow queue are full, the worker rejects the request and the Frontend returns the configured overload status, 529 by default.
Environment variable: DYN_ENGINE_REQUEST_LIMIT
Advanced overflow-queue size for requests waiting in Dynamo before entering the engine. Applies only
when --engine-request-limit or DYN_ENGINE_REQUEST_LIMIT is set. Must be at least 2; the
effective per-worker cap is the engine limit plus this queue limit.
Listen address for the system status server on DYN_SYSTEM_PORT, which serves health, metrics,
and self-hosted model metadata. 0.0.0.0 accepts IPv4 connections only; on IPv6-only hosts, set
::. For a wildcard address, Dynamo advertises a non-loopback address from the same family; ::
also accepts IPv4 connections, so it falls back to a non-loopback IPv4 address. If none exists,
Dynamo advertises loopback and logs a warning; remote callers such as the Frontend cannot reach
that address.