Runtime Configuration (DynamoRuntimeConfig)
Runtime Configuration (DynamoRuntimeConfig)
Field reference for the cross-cutting Dynamo runtime CLI flags and environment variables shared by every backend and the frontend.
DynamoRuntimeConfig holds the configuration that is common to every Dynamo process — the frontend, the standalone router, and all three backend workers (vLLM, SGLang, TRT-LLM) parse this same argument group. Most fields, types, defaults, and choices on this page come from the DynamoRuntimeArgGroup and DynamoRuntimeConfig definitions. The page also catalogs runtime environment-only controls implemented below the Python argument layer.
These are the shared Dynamo runtime flags. Each backend also has its own backend-specific flags: vLLM Configuration, SGLang Configuration, and TensorRT-LLM Configuration. This page covers neither native engine arguments nor backend-specific DYN_* prefixed flags — only the flags from DynamoRuntimeArgGroup.
How the config is loaded
Unless a field is marked environment-only, it has both a CLI flag and an environment variable. The CLI flag takes precedence; the environment variable is the fallback. Boolean fields are negatable — --dyn-enable-structural-tag sets it on, --no-dyn-enable-structural-tag sets it off.
Kubernetes
Local
Set flags in any component container’s args and environment variables in its env, under a service of a DynamoGraphDeployment (DGD). To set a DYN_* variable across all components at once, use the graph-level spec.envs field.
On Kubernetes, several of these variables are auto-injected by the Dynamo operator into every container — see the individual field notes below.
Model discovery and namespace
Dynamo namespace that scopes service discovery. All components in a deployment must share the same namespace to find each other. If DYN_NAMESPACE_WORKER_SUFFIX is also set, -{suffix} is automatically appended to the resolved value to support multiple worker pools serving the same model.
On Kubernetes the operator sets this automatically to {k8s_namespace}-{dgd_name}; override only deliberately.
Environment variable: DYN_NAMESPACE
Dynamo endpoint string in dyn://namespace.component.endpoint format, for example dyn://dynamo.backend.generate. When unset, the endpoint address is inferred from the component’s registration in the discovery backend.
Environment variable: DYN_ENDPOINT
Comma-separated list of OpenAI-compatible endpoint types to enable. Use completions alone for models that do not have a chat template. The obsolete alias --dyn-endpoint-types is accepted for backward compatibility.
Environment variable: DYN_ENDPOINT_TYPES
Communication planes
Service discovery backend. kubernetes uses the K8s API; etcd uses a distributed key-value store (configured via ETCD_* env vars such as ETCD_ENDPOINTS); file uses the local filesystem (path from DYN_FILE_KV, defaulting to $TMPDIR/dynamo_store_kv); mem uses an in-memory store suitable for single-process development. See Discovery Plane for design details.
On Kubernetes the operator sets this automatically to kubernetes; override only deliberately.
Environment variable: DYN_DISCOVERY_BACKEND
Maximum number of seconds that an etcd discovery client spends establishing its initial
connection and primary lease. Dynamo retries with exponential backoff until this deadline. Set a
larger value when etcd can take longer to become available during deployment startup or recovery;
set a smaller value when the process should fail fast. Values must be greater than 0. Zero or
invalid values log a warning and fall back to 120.
Transport used to distribute requests from routers to workers. tcp provides the lowest latency and is recommended for production. nats uses NATS messaging. See Request Plane for design details.
Environment variable: DYN_REQUEST_PLANE
Preferred payload codec advertised by every request-plane endpoint served by this process. Clients
select the codec independently for each destination endpoint. They use the destination’s advertised
codec, or json when a legacy destination does not advertise one. Setting this variable does not
force the codec for outbound requests from the process.
Dynamo caches this process-wide value on its first codec lookup. Restart the process after changing it.
Allowed values: json msgpackIPv4 or IPv6 address, or exact network interface name, used to bind and advertise the TCP response
stream server. If an interface name occurs more than once in the operating system’s interface
list, Dynamo uses the last address. Dynamo trims whitespace and treats an empty value as unset.
Unspecified addresses such as 0.0.0.0 and :: are invalid.
Port used by the TCP response stream server. Dynamo trims surrounding whitespace. When the value
is unset, empty, or 0, the operating system assigns a free port. Values must be between 0 and
65535.
Event publishing transport. ZMQ is the default for every discovery backend; select nats
explicitly to use NATS Core. See Event Plane for design details.
Environment variable: DYN_EVENT_PLANE
IPv4 or IPv6 address, or exact network interface name, advertised by direct ZMQ event publishers.
If an interface name occurs more than once in the operating system’s interface list, Dynamo uses
the last address. Dynamo trims whitespace and treats an empty value as unset. Unspecified addresses
such as 0.0.0.0 and :: are invalid.
This variable changes the discovery address, not the listener address. Direct ZMQ continues to bind an IPv4 wildcard, so an IPv6 value does not enable IPv6 listening. This variable does not affect NATS or ZMQ broker mode.
Number of libzmq background I/O threads in the process-wide event-plane context. All event-plane PUB/SUB sockets in a frontend or worker process share this context. This setting does not change the Tokio runtime size or the number of KV indexer threads.
Set a positive integer before starting the process. Dynamo reads the value when it first creates
the shared context; invalid values fail context initialization. Restart the process to change it.
Set 1 to restore the previous single-thread setting. This variable does not affect NATS or ZMQ
contexts created outside the shared event-plane transport.
KV-cache transfer connector. Accepts zero or more values. Deprecated for vLLM — use --kv-transfer-config instead. For TRT-LLM, valid options are nixl, lmcache, kvbm, null, and none.
Environment variable: DYN_CONNECTOR
Parsing
Tool call parser name for the model. When unset, the backend’s default tool call parsing is used. Valid parser names are determined at runtime by the installed dynamo._core extension — run python -m dynamo.vllm --help to see the available choices on your installation.
Environment variable: DYN_TOOL_CALL_PARSER
Reasoning/chain-of-thought parser name for the model. When unset, no reasoning parsing is performed. Valid parser names are determined at runtime by the installed dynamo._core extension — run python -m dynamo.vllm --help to see the available choices on your installation.
Environment variable: DYN_REASONING_PARSER
Thinking mode to apply when a request omits an explicit thinking control. Request-level thinking, reasoning_effort, and controls in chat_template_args or chat_template_kwargs take precedence. When unset, Dynamo preserves the model or chat template’s native default. See Deployment-Level Thinking Default for examples and precedence details.
Environment variable: DYN_DEFAULT_THINKING_MODE
Exclude tool definitions from the chat template when tool_choice='none'. Prevents models from generating unsolicited raw XML tool calls in the content field. This flag controls the Rust-native chat template path; a matching flag in FrontendArgGroup controls the Python processor side independently.
Environment variable: DYN_EXCLUDE_TOOLS_WHEN_TOOL_CHOICE_NONE
Enable structural tag guided decoding for tool calls. When enabled, configure activation scope and parameter schema strictness with --dyn-structural-tag-scope and --dyn-structural-tag-schema.
Environment variable: DYN_ENABLE_STRUCTURAL_TAG
Controls when structural tags are activated. auto activates them for required or named tool_choice, or when any tool has strict=true or parallel_tool_calls is false. always additionally activates them for tool_choice=auto without those conditions. tool_choice=none is unaffected by either setting. Only meaningful when --dyn-enable-structural-tag is set.
Environment variable: DYN_STRUCTURAL_TAG_SCOPE
Controls parameter schema strictness inside structural tags. auto applies the real parameter schema only to tools with strict=true, leaving all other tools syntactically constrained but schema-unconstrained. strict applies the real parameter schema to every tool. Only meaningful when --dyn-enable-structural-tag is set.
Environment variable: DYN_STRUCTURAL_TAG_SCHEMA
Path to a custom Jinja template file to override the model’s default chat template. This template takes precedence over any template found in the model repository.
Environment variable: DYN_CUSTOM_JINJA_TEMPLATE
Multimodal and media output
Capacity of the multimodal embedding cache in GB. Set to 0 to disable the cache entirely. Increase this for workloads with repeated image or audio inputs to avoid redundant re-encoding.
Environment variable: DYN_MULTIMODAL_EMBEDDING_CACHE_CAPACITY_GB
Space-separated list of output modalities for omni or diffusion mode, for example --output-modalities text image. Defaults to text only. Use image, video, or audio when deploying image/video/audio generation models.
Environment variable: DYN_OUTPUT_MODALITIES
Filesystem URL for storing generated images and videos. Accepts a local file:// path or a remote object-storage URL such as s3://bucket/path.
Environment variable: DYN_MEDIA_OUTPUT_FS_URL
Base HTTP URL for rewriting media file paths in API responses, for example http://localhost:8000/media. When unset, raw filesystem paths are returned in the response body.
Environment variable: DYN_MEDIA_OUTPUT_HTTP_URL
Fault tolerance
Maximum number of seconds that a runtime client locally removes a worker from normal routing after
a request-path failure while service discovery propagates the worker state. Set to 0 to disable
local inhibition. The process reads this value once when the first client initializes; restart the
process after changing it. Discovery updates remain authoritative, and direct dispatch bypasses the
local inhibited set while the selected worker remains in discovery.
Operations
Dump the fully resolved configuration to the specified file path at startup. Useful for auditing the effective values after all env-var and CLI overrides have been applied.
Environment variable: DYN_DUMP_CONFIG_TO
Override the runtime health-check canary payload used by the unified backend’s Worker. Accepts a JSON object string, for example '{"token_ids": [1], "stop_conditions": {"max_tokens": 1}}', or a file reference prefixed with @, for example @/path/to/payload.json. Takes precedence over the engine’s default health_check_payload(). Applies to the unified backend only.
Environment variable: DYN_HEALTH_CHECK_PAYLOAD
Maximum requests handled concurrently by the engine. Setting a positive integer enables worker-side admission control. When the engine slots and the Dynamo overflow queue are full, the worker rejects the request and the Frontend returns the configured overload status, 529 by default.
Environment variable: DYN_ENGINE_REQUEST_LIMIT
Advanced overflow-queue size for requests waiting in Dynamo before entering the engine. Applies only
when --engine-request-limit or DYN_ENGINE_REQUEST_LIMIT is set. Must be at least 2; the
effective per-worker cap is the engine limit plus this queue limit.
Related pages
Dynamo wrapper flag reference for the vLLM backend (DYN_VLLM_*).
Dynamo wrapper flag reference for the SGLang backend (DYN_SGL_*).
Dynamo wrapper flag reference for the TensorRT-LLM backend.
Configuration reference for the Dynamo HTTP frontend component.