> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# DynoSim Replay CLI Reference

`aisimulate predict --stack dynamo` evaluates one concrete workload and deployment configuration
through the Dynamo simulation stack. AISimulate owns the traffic and engine schema. The `ai-dynamo`
package supplies the Dynamo runner and the optional `router` and `planner` configuration adapters.

For an end-to-end workflow, see
[Run a DynoSim Simulation](/dynamo/knowledge-base/concepts/simulation/simulation-runs). To
search configuration domains, see
[Sweep DynoSim Configurations](/dynamo/knowledge-base/concepts/simulation/simulation-sweeps).

> **Warning**
>
> The former Replay `online` mode has no replacement in the unified AISimulate CLI yet. `aisimulate
> predict` and `aisimulate recommend` are offline-only. The separate `python3 -m dynamo.mocker`
> command remains available for launching live workers but does not provide replay orchestration.

## Command

```bash
aisimulate predict --stack dynamo --config prediction.yaml
```

**`-c, --config`** `path` — required

YAML prediction configuration. Prediction files contain concrete values and reject search
domains, presets, `optimization`, and `optimizer`.

---

**`--stack`** `string` — default: engine

Execution stack. Set this option to `dynamo` to load the Dynamo runner and the Router and Planner
adapters registered by `ai-dynamo`.

---

**`--set`** `PATH=YAML_VALUE` — default: null

Override a schema-valid configuration path after loading the YAML file. Repeat the option for
multiple overrides. Values are parsed as YAML, and later assignments win. Sequence indexes are
not supported.

---

**`--output-dir`** `path` — default: ./aisimulate-output

Directory for `prediction.json` and optional per-request output.

---

**`--overwrite`** `flag` — default: false

Replace known AISimulate output files in an existing nonempty output directory. Unrelated files
are preserved.

---

**`--format`** `string` — default: table

Standard-output format.

Allowed values:

table

json

---

**`--capture-per-request`** `flag` — default: false

Write one record per request to `requests.jsonl`.

---

## Configuration Sections

The YAML document uses these top-level sections:

| Section      | Required | Owner          | Purpose                                                                                 |
| ------------ | -------- | -------------- | --------------------------------------------------------------------------------------- |
| `traffic`    | No       | AISimulate     | Request source, load pattern, and stopping condition                                    |
| `engine`     | Yes      | AISimulate     | Model, backend, hardware, topology, worker parallelism, scheduler, KV cache, and timing |
| `router`     | No       | Dynamo adapter | Round-robin or KV-aware routing                                                         |
| `planner`    | No       | Dynamo adapter | Planner policy and scaling behavior                                                     |
| `evaluation` | No       | AISimulate     | Service-level objective (SLO) thresholds used for reporting                             |

Unknown fields are rejected. `predict` accepts only concrete values. Use `aisimulate recommend` for
`choices`, `range`, or `preset` domains.

## Dynamo Prediction Example

```yaml
traffic:
  source:
    type: synthetic
    input_tokens: 1024
    output_tokens: 128
  load:
    type: constant_rate
    requests_per_second: 8
  stop:
    requests: 100
engine:
  mode: aggregated
  model: meta-llama/Meta-Llama-3.1-8B-Instruct
  hardware: h200_sxm
  backend: vllm
  context_length: max
  workers:
    aggregated:
      parallelism:
        replicas: 2
        tensor: 1
        pipeline: 1
        attention_data: 1
        moe_tensor: 1
        moe_expert: 1
      scheduler:
        max_batched_tokens: 8192
        max_sequences: 256
      kv_cache:
        block_size: 64
        prefix_caching: true
        capacity: {type: default, memory_fraction: 0.9}
      timing: {type: default}
      startup_seconds: 0
router:
  policy: round_robin
  prefill_load_model: {type: none}
planner:
  policy: disabled
evaluation:
  sla: {ttft_ms: 500, itl_ms: 50}
```

## Traffic Rules

`traffic` contains `source`, `load`, and `stop` mappings.

* Omitting the entire section creates 100 independent synthetic requests at concurrency 10, with
  1,024 input tokens and 128 output tokens per request.
* `source.type: synthetic` creates independent requests with `input_tokens` and `output_tokens`.
* `source.type: synthetic-session` creates ordered multi-turn sessions.
* `source.type: trace` reads one or more paths. Supported formats include `mooncake`,
  `mooncake-delta`, `agentic_mooncake`, `applied_compute_agentic`, and `dynamo`.
* `load.type: concurrency` keeps a fixed number of requests or sessions active.
* `load.type: constant_rate` schedules evenly spaced arrivals from `requests_per_second` or
  `sessions_per_second`.
* `load.type: poisson` uses the matching rate with exponential inter-arrival times and an optional
  `seed`, which defaults to 42.
* `load.type: trace_timestamps` preserves trace timing and accepts a positive `speedup`.
* `stop.requests` applies to independent requests. `stop.sessions` applies to session sources.
* `stop.requests_per_load_unit` and `stop.sessions_per_load_unit` derive the count from the concrete
  concurrency or arrival rate.
* `stop.max_virtual_time_seconds` is a soft virtual-time cutoff. Requests admitted before the
  cutoff may finish afterward.

For a Dynamo request trace, omit `traffic.source.block_size`; AISimulate derives it from the trace
records and rejects mixed block sizes. Dynamo format accepts multiple trace shards. Other formats
accept exactly one path, and their block size defaults to 512 when omitted.

`mooncake-delta` and `agentic_mooncake` require aggregated engine mode.
`agentic_mooncake` also requires `trace_timestamps` and rejects the virtual-time cutoff.
`applied_compute_agentic` requires concurrency load.
With the Dynamo stack, omit `planner` or set `planner.policy: disabled` for `mooncake-delta`,
`agentic_mooncake`, and Dynamo traces that carry `agent_context` records.

## Engine and Adapter Rules

* `engine.mode: aggregated` requires `engine.workers.aggregated`.
* `engine.mode: disaggregated` requires `engine.workers.prefill`,
  `engine.workers.decode`, and `engine.kv_transfer`.
* `aisimulate predict` and `aisimulate recommend` currently support TensorRT-LLM only in
  aggregated mode. This is an offline simulation limitation; Dynamo runtime deployments support
  TensorRT-LLM disaggregated serving.
* Each `parallelism` mapping is concrete and contains `replicas`, `tensor`, `pipeline`,
  `attention_data`, `moe_tensor`, and `moe_expert`.
* `engine.context_length` defaults to `max`, which AISimulate resolves from the model's Hugging Face
  configuration. Default KV block sizes are 64 for vLLM, 1 for SGLang, and 32 for TensorRT-LLM.
* Scheduler defaults are 8,192 batched tokens for every role and 256 sequences for aggregated and
  decode workers. Prefill workers default to one sequence.
* `timing.type: default` uses the AIConfigurator forward-pass model shipped in the `aisimulate`
  wheel. `fixed` requires both `prefill_ms` and `decode_ms`; `polynomial` selects the built-in
  polynomial model.
* `router.policy: kv_router` requires more than one routable worker. Set
  `router.prefill_load_model.type` to `none` or `aic`.
* `planner.policy` is `disabled` or `enabled`. When enabled, `planner.max_num_gpus` limits the
  Planner runtime budget; it is distinct from recommendation candidate constraints.
* `evaluation.sla` accepts either `e2e_ms` alone or `ttft_ms` and `itl_ms` together. The two forms
  are mutually exclusive.

## Overrides

`--set` accepts dot-separated, schema-valid paths. The field does not need to be present in the
input YAML:

```bash
aisimulate predict \
  --stack dynamo \
  --config prediction.yaml \
  --set engine.workers.aggregated.parallelism.replicas=4 \
  --set router.policy=kv_router
```

An override cannot create an unknown field. Override a complete mapping when changing a tagged
configuration shape, such as `traffic.source`.

## Output

The output directory contains:

```text
aisimulate-output/
├── prediction.json
└── requests.jsonl
```

`prediction.json` preserves the Dynamo runner report, including summary metrics and available
Planner diagnostics. `requests.jsonl` is present only with `--capture-per-request`. `--format`
changes standard output but not durable files.

The command exits with `0` on success, `1` for execution failure, `2` for CLI or configuration
errors, and `130` when interrupted.