> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# DynoSim Replay CLI Reference

`aisimulate predict --stack dynamo` evaluates one concrete workload and deployment configuration
through the Dynamo simulation stack. AISimulate owns the traffic and engine schema. The `ai-dynamo`
package supplies the Dynamo runner and the optional `router` and `planner` configuration adapters.

For an end-to-end workflow, see
[Run a DynoSim Simulation](/dynamo/dev/knowledge-base/concepts/simulation/simulation-runs). To
search configuration domains, see
[Sweep DynoSim Configurations](/dynamo/dev/knowledge-base/concepts/simulation/simulation-sweeps).

The former Replay `online` mode has no replacement in the unified AISimulate CLI yet. `aisimulate
predict` and `aisimulate recommend` are offline-only. The separate `python3 -m dynamo.mocker`
command remains available for launching live workers but does not provide replay orchestration.

## Command

```bash
aisimulate predict --stack dynamo --config prediction.yaml
```

**`-c, --config`** `path` — required

YAML prediction configuration. Prediction files contain concrete values and reject search
domains, presets, `optimization`, and `optimizer`.

---

**`--stack`** `string` — default: engine

Execution stack. Set this option to `dynamo` to load the Dynamo runner and the Router and Planner
adapters registered by `ai-dynamo`.

---

**`--set`** `PATH=YAML_VALUE` — default: null

Override a schema-valid configuration path after loading the YAML file. Repeat the option for
multiple overrides. Values are parsed as YAML, and later assignments win. Sequence indexes are
not supported.

---

**`--output-dir`** `path` — default: ./aisimulate-output

Directory for `prediction.json` and optional per-request output.

---

**`--overwrite`** `flag` — default: false

Replace known AISimulate output files in an existing nonempty output directory. Unrelated files
are preserved.

---

**`--format`** `string` — default: table

Standard-output format.

Allowed values:

table

json

---

**`--capture-per-request`** `flag` — default: false

Write one record per request to `requests.jsonl`.

---

## Configuration Sections

The YAML document uses these top-level sections:

| Section      | Required | Owner          | Purpose                                                                                 |
| ------------ | -------- | -------------- | --------------------------------------------------------------------------------------- |
| `traffic`    | No       | AISimulate     | Request source, load pattern, and stopping condition                                    |
| `engine`     | Yes      | AISimulate     | Model, backend, hardware, topology, worker parallelism, scheduler, KV cache, and timing |
| `router`     | No       | Dynamo adapter | Round-robin or KV-aware routing                                                         |
| `planner`    | No       | Dynamo adapter | Planner policy and scaling behavior                                                     |
| `evaluation` | No       | AISimulate     | Service-level objective (SLO) thresholds used for reporting                             |

Unknown fields are rejected. `predict` accepts only concrete values. Use `aisimulate recommend` for
`choices`, `range`, or `preset` domains.

## Dynamo Prediction Example

```yaml
traffic:
  source:
    type: synthetic
    input_tokens: 1024
    output_tokens: 128
  load:
    type: constant_rate
    requests_per_second: 8
  stop:
    requests: 100
engine:
  mode: aggregated
  model: meta-llama/Meta-Llama-3.1-8B-Instruct
  hardware: h200_sxm
  backend: vllm
  context_length: max
  workers:
    aggregated:
      parallelism:
        replicas: 2
        tensor: 1
        pipeline: 1
        attention_data: 1
        moe_tensor: 1
        moe_expert: 1
      scheduler:
        max_batched_tokens: 8192
        max_sequences: 256
      kv_cache:
        block_size: 64
        prefix_caching: true
        capacity: {type: default, memory_fraction: 0.9}
      timing: {type: default}
      startup_seconds: 0
router:
  policy: round_robin
  prefill_load_model: {type: none}
planner:
  policy: disabled
evaluation:
  sla: {ttft_ms: 500, itl_ms: 50}
```

## Traffic Rules

`traffic` contains `source`, `load`, and `stop` mappings.

* Omitting the entire section creates 100 independent synthetic requests at concurrency 10, with
  1,024 input tokens and 128 output tokens per request.
* `source.type: synthetic` creates independent requests with `input_tokens` and `output_tokens`.
* `source.type: synthetic-session` creates ordered multi-turn sessions.
* `source.type: trace` reads one or more paths. Supported formats include `mooncake`,
  `mooncake-delta`, `agentic_mooncake`, `applied_compute_agentic`, and `dynamo`.
* `load.type: concurrency` keeps a fixed number of requests or sessions active.
* `load.type: constant_rate` schedules evenly spaced arrivals from `requests_per_second` or
  `sessions_per_second`.
* `load.type: poisson` uses the matching rate with exponential inter-arrival times and an optional
  `seed`, which defaults to 42.
* `load.type: trace_timestamps` preserves trace timing and accepts a positive `speedup`.
* `stop.requests` applies to independent requests. `stop.sessions` applies to session sources.
* `stop.requests_per_load_unit` and `stop.sessions_per_load_unit` derive the count from the concrete
  concurrency or arrival rate.
* `stop.max_virtual_time_seconds` is a soft virtual-time cutoff. Requests admitted before the
  cutoff may finish afterward.

For a Dynamo request trace, omit `traffic.source.block_size`; AISimulate derives it from the trace
records and rejects mixed block sizes. Dynamo format accepts multiple trace shards. Other formats
accept exactly one path, and their block size defaults to 512 when omitted.

`mooncake-delta` and `agentic_mooncake` require aggregated engine mode.
`agentic_mooncake` also requires `trace_timestamps` and rejects the virtual-time cutoff.
`applied_compute_agentic` requires concurrency load.
With the Dynamo stack, omit `planner` or set `planner.policy: disabled` for `mooncake-delta`,
`agentic_mooncake`, and Dynamo traces that carry `agent_context` records.

### Typed agentic replay through the Dynamo API

The lower-level `dynamo.replay.run_trace_replay(...)` API additionally accepts `weka` and supports
Agentic Mooncake, Weka, and fully agentic Dynamo traces in aggregated or disaggregated offline mode.
These API-only inputs are not yet fields in the `aisimulate predict` YAML schema.

Set `agentic_lanes` to a positive integer to replay a fixed number of trajectories concurrently.
Plays are stable-sorted and stride-assigned to lanes. A lane starts its next play only after its
current play is quiescent, and lanes do not steal work. Agentic replay rejects
`replay_concurrency`, Planner scaling, and mixed-model Weka corpora.

```python
from dynamo.mocker import MockEngineArgs
from dynamo.replay import run_trace_replay

report = run_trace_replay(
    trace_files="traces/weka-agentx",
    trace_format="weka",
    agentic_lanes=12,
    router_mode="kv_router",
    num_workers=4,
    extra_engine_args=MockEngineArgs(engine_type="vllm", block_size=64),
)
```

Agentic Mooncake v2 begins with a required versioned header:

```json
{"schema":"dynamo.agentic_mooncake","version":2,"block_size":64,"hash_id_scope":"local","source":{"format":"weka","digest":"<corpus-digest>"}}
```

Each request row contains a globally unique `request_id`, nonempty `play_id`, `session_id`, `model`,
exact input and output metadata, `hash_ids`, and `not_before_ms`. The optional `dependencies` array
contains typed incoming edges. Each edge identifies its predecessor, a `dispatch` or `completion`
trigger, a nonnegative delay, and a `sequence`, `spawn`, `join`, or `replay_barrier` relation.

The Weka importer recursively traverses local files in deterministic order, namespaces local
identity by normalized source-relative path, and preserves public AgentX semantics. Request and
nested-subagent `t` values are absolute seconds from the trace start, and every request requires a
finite `api_time`. For each explicit subagent marker, the latest preceding outer request in source
order becomes the inferred owner. Spawn and blocking-join edges stay within that request's
reconstructed stream. Weka does not record an explicit parent request ID, so this rule is a
deterministic replay policy rather than captured provider causality.

Sequential turns wait for completion. Overlapping subagents depend on parent dispatch,
post-completion subagents depend on parent completion, blocking results join the owning parent
stream, and `async_launched` work remains background. Timestamp-inferred cross-stream ordering uses
`replay_barrier`, not a causal join. Inner requests before their subagent marker, unsupported
terminal states, blocking empty subagents, symlinks, mixed block sizes, and mixed models are
rejected.

To materialize the same validated graph as canonical Agentic Mooncake v2 JSONL:

```bash
cargo run -p dynamo-bench \
  --features weka-to-agentic-mooncake \
  --bin weka_to_agentic_mooncake -- \
  --input traces/weka-agentx \
  --output /tmp/agentic-mooncake-v2.jsonl
```

The converter verifies that direct Weka ingestion and reparsed v2 produce the same canonical graph
identity. Weka remains an input format; Dynamo does not integrate with the AIPerf runtime or export
Weka.

## Engine and Adapter Rules

* `engine.mode: aggregated` requires `engine.workers.aggregated`.
* `engine.mode: disaggregated` requires `engine.workers.prefill`,
  `engine.workers.decode`, and `engine.kv_transfer`.
* `aisimulate predict` and `aisimulate recommend` currently support TensorRT-LLM only in
  aggregated mode. This is an offline simulation limitation; Dynamo runtime deployments support
  TensorRT-LLM disaggregated serving.
* Each `parallelism` mapping is concrete and contains `replicas`, `tensor`, `pipeline`,
  `attention_data`, `moe_tensor`, and `moe_expert`.
* `engine.context_length` defaults to `max`, which AISimulate resolves from the model's Hugging Face
  configuration. Default KV block sizes are 64 for vLLM, 1 for SGLang, and 32 for TensorRT-LLM.
* Scheduler defaults are 8,192 batched tokens for every role and 256 sequences for aggregated and
  decode workers. Prefill workers default to one sequence.
* `timing.type: default` uses the AIConfigurator forward-pass model shipped in the `aisimulate`
  wheel. `fixed` requires both `prefill_ms` and `decode_ms`; `polynomial` selects the built-in
  polynomial model.
* `router.policy: kv_router` requires more than one routable worker. Set
  `router.prefill_load_model.type` to `none` or `aic`.
* `planner.policy` is `disabled` or `enabled`. When enabled, `planner.max_num_gpus` limits the
  Planner runtime budget; it is distinct from recommendation candidate constraints.
* `evaluation.sla` accepts either `e2e_ms` alone or `ttft_ms` and `itl_ms` together. The two forms
  are mutually exclusive.

## Overrides

`--set` accepts dot-separated, schema-valid paths. The field does not need to be present in the
input YAML:

```bash
aisimulate predict \
  --stack dynamo \
  --config prediction.yaml \
  --set engine.workers.aggregated.parallelism.replicas=4 \
  --set router.policy=kv_router
```

An override cannot create an unknown field. Override a complete mapping when changing a tagged
configuration shape, such as `traffic.source`.

## Output

The output directory contains:

```text
aisimulate-output/
├── prediction.json
└── requests.jsonl
```

`prediction.json` preserves the Dynamo runner report, including summary metrics and available
Planner diagnostics. `requests.jsonl` is present only with `--capture-per-request`. `--format`
changes standard output but not durable files.

The command exits with `0` on success, `1` for execution failure, `2` for CLI or configuration
errors, and `130` when interrupted.