DynoSim Replay CLI Reference

aisimulate predict --stack dynamo evaluates one concrete workload and deployment configuration through the Dynamo simulation stack. AISimulate owns the traffic and engine schema. The ai-dynamo package supplies the Dynamo runner and the optional router and planner configuration adapters.

For an end-to-end workflow, see Run a DynoSim Simulation. To search configuration domains, see Sweep DynoSim Configurations.

Command

aisimulate predict --stack dynamo --config prediction.yaml

Configuration Sections

The YAML document uses these top-level sections:

Unknown fields are rejected. predict accepts only concrete values. Use aisimulate recommend for choices, range, or preset domains.

Dynamo Prediction Example

traffic:
  source:
    type: synthetic
    input_tokens: 1024
    output_tokens: 128
  load:
    type: constant_rate
    requests_per_second: 8
  stop:
    requests: 100
engine:
  mode: aggregated
  model: meta-llama/Meta-Llama-3.1-8B-Instruct
  hardware: h200_sxm
  backend: vllm
  context_length: max
  workers:
    aggregated:
      parallelism:
        replicas: 2
        tensor: 1
        pipeline: 1
        attention_data: 1
        moe_tensor: 1
        moe_expert: 1
      scheduler:
        max_batched_tokens: 8192
        max_sequences: 256
      kv_cache:
        block_size: 64
        prefix_caching: true
        capacity: {type: default, memory_fraction: 0.9}
      timing: {type: default}
      startup_seconds: 0
router:
  policy: round_robin
  prefill_load_model: {type: none}
planner:
  policy: disabled
evaluation:
  sla: {ttft_ms: 500, itl_ms: 50}

Traffic Rules

traffic contains source, load, and stop mappings.

• Omitting the entire section creates 100 independent synthetic requests at concurrency 10, with 1,024 input tokens and 128 output tokens per request.
• source.type: synthetic creates independent requests with input_tokens and output_tokens.
• source.type: synthetic-session creates ordered multi-turn sessions.
• source.type: trace reads one or more paths. Supported formats include mooncake, mooncake-delta, agentic_mooncake, applied_compute_agentic, and dynamo.
• load.type: concurrency keeps a fixed number of requests or sessions active.
• load.type: constant_rate schedules evenly spaced arrivals from requests_per_second or sessions_per_second.
• load.type: poisson uses the matching rate with exponential inter-arrival times and an optional seed, which defaults to 42.
• load.type: trace_timestamps preserves trace timing and accepts a positive speedup.
• stop.requests applies to independent requests. stop.sessions applies to session sources.
• stop.requests_per_load_unit and stop.sessions_per_load_unit derive the count from the concrete concurrency or arrival rate.
• stop.max_virtual_time_seconds is a soft virtual-time cutoff. Requests admitted before the cutoff may finish afterward.

For a Dynamo request trace, omit traffic.source.block_size; AISimulate derives it from the trace records and rejects mixed block sizes. Dynamo format accepts multiple trace shards. Other formats accept exactly one path, and their block size defaults to 512 when omitted.

mooncake-delta and agentic_mooncake require aggregated engine mode. agentic_mooncake also requires trace_timestamps and rejects the virtual-time cutoff. applied_compute_agentic requires concurrency load. With the Dynamo stack, omit planner or set planner.policy: disabled for mooncake-delta, agentic_mooncake, and Dynamo traces that carry agent_context records.

Engine and Adapter Rules

• engine.mode: aggregated requires engine.workers.aggregated.
• engine.mode: disaggregated requires engine.workers.prefill, engine.workers.decode, and engine.kv_transfer.
• aisimulate predict and aisimulate recommend currently support TensorRT-LLM only in aggregated mode. This is an offline simulation limitation; Dynamo runtime deployments support TensorRT-LLM disaggregated serving.
• Each parallelism mapping is concrete and contains replicas, tensor, pipeline, attention_data, moe_tensor, and moe_expert.
• engine.context_length defaults to max, which AISimulate resolves from the model’s Hugging Face configuration. Default KV block sizes are 64 for vLLM, 1 for SGLang, and 32 for TensorRT-LLM.
• Scheduler defaults are 8,192 batched tokens for every role and 256 sequences for aggregated and decode workers. Prefill workers default to one sequence.
• timing.type: default uses the AIConfigurator forward-pass model shipped in the aisimulate wheel. fixed requires both prefill_ms and decode_ms; polynomial selects the built-in polynomial model.
• router.policy: kv_router requires more than one routable worker. Set router.prefill_load_model.type to none or aic.
• planner.policy is disabled or enabled. When enabled, planner.max_num_gpus limits the Planner runtime budget; it is distinct from recommendation candidate constraints.
• evaluation.sla accepts either e2e_ms alone or ttft_ms and itl_ms together. The two forms are mutually exclusive.

Overrides

--set accepts dot-separated, schema-valid paths. The field does not need to be present in the input YAML:

aisimulate predict \
  --stack dynamo \
  --config prediction.yaml \
  --set engine.workers.aggregated.parallelism.replicas=4 \
  --set router.policy=kv_router

An override cannot create an unknown field. Override a complete mapping when changing a tagged configuration shape, such as traffic.source.

Output

The output directory contains:

aisimulate-output/
├── prediction.json
└── requests.jsonl

prediction.json preserves the Dynamo runner report, including summary metrics and available Planner diagnostics. requests.jsonl is present only with --capture-per-request. --format changes standard output but not durable files.

The command exits with 0 on success, 1 for execution failure, 2 for CLI or configuration errors, and 130 when interrupted.