DynoSim Replay CLI Reference

python -m dynamo.replay runs one workload through one simulated Dynamo configuration and emits an AIPerf-style report. This page is the flag and schema reference. For the workflow — input formats, run modes, and when to use DynoSim versus AIPerf — see DynoSim Runs; for searching many configurations, see DynoSim Sweeps.

The command reuses the mocker engine, so engine behavior is configured through JSON passed to --extra-engine-args (or the staged --prefill-engine-args / --decode-engine-args) rather than individual flags. For the underlying engine settings, see the Mocker CLI Reference.

python -m dynamo.replay requires exactly one of: positional trace files, or all of --input-tokens, --output-tokens, and --request-count. The dynamo format accepts one or more trace shards. Every other trace format requires exactly one file.

Workload input

Synthetic workload shaping

Apply only to synthetic (non-trace) runs.

Trace file schema

Mooncake-style JSONL traces contain one request per line. A basic row accepts:

Example:

1
{"timestamp": 0, "input_length": 6755, "output_length": 500, "hash_ids": [0, 1, 2, 3]}

Rows without session_id are independent requests. Priority fields affect only pending requests when --router-mode kv_router is active; they do not change round-robin routing or execution order inside a selected engine.

Multi-turn sessions

Use the same session_id for every turn in one session. Turn n+1 waits for turn n to complete, then applies delay, delay_ms, or the timestamp delta between consecutive rows.

1
{"session_id":"session-a","timestamp":1000,"input_length":2048,"output_length":128,"hash_ids":[1,2,3,4]}
2
{"session_id":"session-a","delay_ms":50,"input_length":2560,"output_length":128,"hash_ids":[1,2,3,4,5]}

Agentic Mooncake traces

Set --trace-format agentic_mooncake to model request-level dependencies. Each row requires a stable request_id and can include:

Rows without wait_for use their timestamp as the start time. Rows with dependencies wait for all listed requests and then apply delay + tool_wait_ms.

Agentic Mooncake is an input format for externally authored Mooncake-compatible traces. It is not an intermediate representation for Dynamo request traces.

Dynamo request traces

Set --trace-format dynamo to read one or more dynamo.request.trace.v1 JSONL or JSONL.GZ shards. DynoSim reads the original records directly and does not create an intermediate Mooncake file. It derives and validates the trace block size across all shards.

Rows without agent_context use the standard request model. If every request contains agent_context, DynoSim reconstructs the agent-aware model, including session dependencies and tool waits. A replay cannot mix context-free and agent-aware request rows. For the complete capture schema, see Observe a Local Deployment.

--trace-block-size defines how many tokens each trace hash_id represents. It is independent from the simulated engine block_size.

Execution

Engine and router JSON

These flags take JSON strings. Engine settings such as block_size, engine_type, dp_size, speedup_ratio, and decode_speedup_ratio belong here, not as top-level flags. Unspecified fields fall back to MockEngineArgs::default() and KvRouterConfig::default().

Requests can set policy_class in Mooncake JSONL rows. A recognized policy family combines with the router-observed uncached Input Sequence Length (ISL) bucket to select a physical queue. An exact explicit class bypasses cache bucketing. Missing, unknown, and ordinary physical-class names use the selected profile’s default_policy_family. For the YAML schema and queue behavior, see Router Configuration.

Engine-timing AIC fields (engine-args JSON)

Set inside --extra-engine-args (or the staged prefill/decode engine args) to drive the simulated worker’s own timing model from AIC. This is the engine-timing AIC surface, distinct from the router-side --aic-* flags below.

Router-side AIC prompt-load flags

Top-level flags used only for router-side prompt-load modeling, together with router_prefill_load_model: "aic" in --router-config. Engine timing AIC still belongs in the engine-args JSON above. Both surfaces accept the MoE parallelism fields; for Kimi-style TP-only MoE configs, set --aic-moe-tp-size equal to --aic-tp-size, with --aic-moe-ep-size 1 and --aic-attention-dp-size 1.

Constraints

DynoSim validates these before running and fails immediately on any violation.

Shared constraints:

• extra_engine_args.engine_type must be vllm, sglang, or trtllm
• aggregated simulation requires the existing aggregated args path
• disaggregated simulation requires both prefill_engine_args and decode_engine_args
• disaggregated simulation requires router_mode=kv_router
• dp_size must be 1
• disaggregated simulation requires matching block_size in prefill_engine_args and decode_engine_args

Additional offline constraints:

• offline kv_router requires num_workers > 1
• single-worker offline mode is a dedicated fast path for vllm, sglang, and trtllm; it supports flat request runs and workload-driven multi-turn runs
• offline disaggregated simulation is a separate two-stage runtime with prefill and decode worker pools

Additional online constraints:

• the current live simulation path is also limited to aggregated workers

Report schema

The report — and the equivalent Python APIs dynamo.replay.run_trace_replay(...) and dynamo.replay.run_synthetic_trace_replay(...) — contains:

• request counts
• input and output token totals
• virtual duration and wall-clock runtime
• request and token throughput
• prefix cache reuse ratio
• TTFT, TTST, TPOT, ITL, and end-to-end latency summaries
• output-token-throughput-per-user summaries

Related pages