Mocker CLI Reference

Command-line flags for the python -m dynamo.mocker simulated worker.

View as Markdown

python -m dynamo.mocker launches a simulated Dynamo worker that registers with the frontend, publishes KV events, and exercises the router and planner paths without a GPU. This page is the flag reference. For the deployment workflow and local/Kubernetes launch recipes, see Simulate a Kubernetes Deployment or Simulate a Local Deployment; for the engine internals these flags configure, see Mocker Engine Architecture.

The same engine settings apply to DynoSim Runs: python -m dynamo.replay reuses the mocker engine and accepts these values as JSON through --extra-engine-args, --prefill-engine-args, and --decode-engine-args rather than as individual flags.

Core and model

--model-path
stringRequired

HuggingFace model ID or local path for the tokenizer.

--model-name
stringDefaults to derived from --model-path

Model name used in API responses.

--endpoint
stringDefaults to auto-derived

Dynamo endpoint string. Defaults are namespace-dependent, and prefill workers use a different default endpoint than aggregated or decode workers.

--engine-type
stringDefaults to vllm

Engine simulation type.

Allowed values: vllm sglang
--extra-engine-args
pathDefaults to null

Path to a JSON file with mocker configuration. Overrides individual CLI arguments.

KV cache

--num-gpu-blocks-override
integerDefaults to 16384

Number of KV cache blocks.

--block-size
integerDefaults to 64 (vllm) / engine-specific

Tokens per KV cache block. For sglang, when omitted, the effective page/block size defaults to 1 or to --sglang-page-size when provided.

--enable-prefix-caching
booleanDefaults to true

Enable prefix caching. Pass --no-enable-prefix-caching to disable.

--kv-cache-dtype
stringDefaults to auto

KV cache dtype for the bytes-per-token computation.

--kv-bytes-per-token
integerDefaults to auto-computed

KV cache bytes per token. Overrides the auto-computation.

Scheduling

--max-num-seqs
integerDefaults to 256

Maximum concurrent sequences.

--max-num-batched-tokens
integerDefaults to 8192

Maximum tokens per batch.

--enable-chunked-prefill
booleanDefaults to true

Enable chunked prefill. Pass --no-enable-chunked-prefill to disable.

--preemption-mode
stringDefaults to lifo

Decode eviction policy under memory pressure. lifo is vLLM v1 style.

Allowed values: lifo fifo

Timing

--speedup-ratio
floatDefaults to 1.0

Timing speedup factor applied to simulated prefill and decode durations.

--decode-speedup-ratio
floatDefaults to 1.0

Decode-only speedup multiplier, for example for Eagle speculation.

--startup-time
floatDefaults to null

Simulated startup delay, in seconds.

Data parallelism and workers

--data-parallel-size
integerDefaults to 1

Number of DP replicas.

--num-workers
integerDefaults to 1

Workers per process. Prefer this over launching many separate mocker processes: all workers share one tokio runtime and thread pool.

--stagger-delay
floatDefaults to -1 (auto)

Delay between worker launches, in seconds. 0 disables; -1 enables auto mode.

Performance modeling

--planner-profile-data
pathDefaults to null

Path to either a mocker-format .npz file or a profiler results directory.

--reasoning
JSON stringDefaults to null

JSON config for emitting reasoning token spans, with start_thinking_token_id, end_thinking_token_id, and thinking_ratio.

AIC performance model

Opt-in flags for the NVIDIA AI Configurator (AIC) latency model. Default mocker runs do not use AIC. For the timing-model design, see Mocker Engine Architecture.

--aic-perf-model
booleanDefaults to false

Use the AIC SDK for latency prediction instead of the interpolated or polynomial models. Opt-in only: default mocker and DynoSim run paths do not use AIC. Requires aiconfigurator installed and usable AIC systems/perf data for the requested system/backend/version tuple.

--aic-system
stringDefaults to h200_sxm

AIC system name, for example h200_sxm. Used with --aic-perf-model.

--aic-backend-version
stringDefaults to auto

AIC backend engine version, for example 0.12.0 for vLLM. When unset, uses the default version for the backend.

--aic-tp-size
integerDefaults to 1

Tensor-parallel size for AIC latency prediction. Affects only AIC performance-model lookups, not mocker scheduling.

--aic-moe-tp-size
integerDefaults to null

Mixture-of-Experts tensor-parallel size for AIC latency prediction. Required by some MoE models.

--aic-moe-ep-size
integerDefaults to null

Mixture-of-Experts expert-parallel size for AIC latency prediction. Required by some MoE models.

--aic-attention-dp-size
integerDefaults to null

Attention data-parallel size for AIC latency prediction. Required by some MoE models.

Disaggregation

--disaggregation-mode
stringDefaults to agg

Worker mode.

Allowed values: agg prefill decode
--bootstrap-ports
stringDefaults to null

Comma-separated rendezvous base ports, one per worker in disaggregated mode.

--kv-transfer-bandwidth
floatDefaults to 64.0

KV cache transfer bandwidth in GB/s. Set to 0 to disable.

Event and request transport

--event-plane
stringDefaults to env-driven (nats)

Event transport.

Allowed values: nats zmq
--request-plane
stringDefaults to env-driven (tcp)

Request transport.

Allowed values: nats tcp
--discovery-backend
stringDefaults to env-driven (etcd)

Discovery backend.

Allowed values: kubernetes etcd file mem
--zmq-kv-events-ports
stringDefaults to null

Comma-separated ZMQ PUB base ports for KV event publishing, one per worker.

--zmq-replay-ports
stringDefaults to null

Comma-separated ZMQ ROUTER base ports for gap recovery, one per worker.

SGLang-specific

Apply only when --engine-type sglang.

--sglang-schedule-policy
stringDefaults to fifo / fcfs

SGLang scheduling policy. fifo/fcfs is the default; lpm is longest prefix match.

Allowed values: fifo fcfs lpm
--sglang-page-size
integerDefaults to 1

SGLang radix-cache page size in tokens. Also becomes the effective block size when --engine-type sglang and --block-size is omitted.

--sglang-max-prefill-tokens
integerDefaults to 16384

SGLang max prefill-token budget per batch.

--sglang-chunked-prefill-size
integerDefaults to 8192

SGLang chunked-prefill chunk size.

--sglang-clip-max-new-tokens
integerDefaults to 4096

SGLang admission-budget cap for max new tokens.

--sglang-schedule-conservativeness
floatDefaults to 1.0

SGLang schedule conservativeness factor.

Environment variables

VariableDefaultDescription
DYN_MOCKER_KV_CACHE_TRACEoffSet to 1 or true to log structured KV cache allocation and eviction traces.
Simulate a Kubernetes Deployment

Deploy and run the Mocker worker on Kubernetes.

Simulate a Local Deployment

Run a frontend and Mocker worker from the command line.

Run a DynoSim Simulation

Run one workload through a simulated configuration with python -m dynamo.replay.

Mocker Engine Architecture

Scheduler, KV block manager, eviction, and timing internals.

Planner Configuration

The PlannerConfig field reference for the Dynamo Planner service.