> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt. # DynoSim Architecture DynoSim connects a workload driver to one or more Mocker engine cores and records request and token timing for analysis. The unified `aisimulate predict` and `aisimulate recommend` commands run this simulation offline. The former public Replay online CLI is unavailable, although the Python replay SDK retains online mode. The separate `python3 -m dynamo.mocker` command launches live Mocker workers without replay orchestration. For task-oriented instructions, see [Run a DynoSim Simulation](/dynamo/knowledge-base/concepts/simulation/simulation-runs), [Sweep DynoSim Configurations](/dynamo/knowledge-base/concepts/simulation/simulation-sweeps), and [Benchmark Planner Decisions](/dynamo/knowledge-base/concepts/simulation/planner-replay). For engine-core details, see [Mocker Engine Architecture](/dynamo/knowledge-base/modular-components/backends/mocker/mocker-engine-architecture). ## Replay harness ```mermaid flowchart LR LD["Trace or synthetic load driver"] --> H["DynoSim replay harness"] H --> SES["Single-engine simulation"] H --> MES["Multi-engine simulation"] SES --> H MES --> H H --> TC["Trace collector and reports"] ``` The load driver supplies either a trace or a generated workload. The harness admits requests into a simulated configuration, advances the simulation, and passes lifecycle timing to the trace collector. The collector produces the AIPerf-style terminal summary and JSON report. Aggregated simulation uses one event loop for single-worker, multi-worker, and attention-DP deployments. Disaggregated simulation uses separate prefill and decode pools. The same replay boundary also supports KV routing and Planner-in-the-loop experiments. ## Trace ingestion and session reconstruction ```mermaid flowchart LR M["Mooncake-compatible JSONL"] --> MP["Mooncake parser"] D["Dynamo request-trace shards"] --> DP["dynamo.request.trace.v1 parser"] MP --> N["Normalized replay workload"] DP --> B{"agent_context on every request?"} B -->|No| S["Standard request model"] B -->|Yes| A["Agent-aware session model"] S --> N A --> N N --> LD["Load driver"] ``` Mooncake-compatible formats carry request timing, token lengths, prefix hashes, and optional session or dependency fields. Dynamo request traces are loaded directly from one or more JSONL or JSONL.GZ shards. The loader maps Dynamo's sequence-aware hashes to compact replay IDs without writing an intermediate Mooncake file and validates that every shard uses the same embedded trace block size. Context-free Dynamo records become independent requests. When every request contains `agent_context`, the loader reconstructs session dependencies and tool waits. It rejects mixed traces instead of silently dropping agent relationships. This path preserves session identity exported by supported agent harnesses, including parent and child sessions. ## Component composition ```mermaid flowchart TD subgraph CORE["Mocker engine core"] S["Scheduler model"] --> F["Forward-pass timing model"] S --> K["KV block manager"] end T["Prefill/decode KV handoff simulation"] R["KV router simulation"] P["Planner simulation adapter"] SES["Single-engine simulation"] MES["Multi-engine simulation"] SES --> CORE MES --> CORE MES --> T MES --> R MES --> P ``` The engine core owns scheduling, KV allocation, prefix caching, preemption, and forward-pass timing. The multi-engine layer adds behavior that requires coordination across engine instances. ## Execution model Offline execution drives Mocker engine cores directly. It uses a logical clock and does not require a frontend, worker registration, etcd, NATS, or HTTP traffic. This path is appropriate for fast, repeatable configuration comparisons and continuous-integration tests. Run `python3 -m dynamo.mocker` for the supported live worker CLI. The Python replay SDK retains online mode for programmatic callers, but no public Replay CLI currently exposes online replay orchestration. ## Routing simulation Round-robin simulation assigns requests without KV-aware scoring. KV-router simulation layers an in-process indexer, worker queues, and routing lifecycle events over the engine cores. Router queueing uses simulation time in offline mode. The router observes request admission, prefill completion, and sequence release. It can estimate prompt-side load from token counts or an AIConfigurator timing model. These estimates influence worker selection but do not replace the engine scheduler's own queue and KV-cache behavior. Policy-class replay uses the same policy-family and cache-bucket model as the live router: ```mermaid flowchart LR R["Replay request"] --> C{"policy_class"} C -->|Exact explicit class| Q["Physical policy queue"] C -->|Known family| B["Observed uncached-ISL bucket"] C -->|Missing or unknown| F["default_policy_family"] F --> B B --> Q Q --> D["Deficit round-robin dispatch"] ``` The trace loader preserves `policy_class` metadata for the replay runtime. The unified public YAML does not expose the former startup policy-file and `--model-name` CLI controls. ## Planner simulation adapter Planner-in-the-loop simulation supplies traffic observations from the replay harness instead of Prometheus. On each Planner traffic tick, the adapter reports: | Replay metric | Planner meaning | |---|---| | `num_req` | Completed requests in the observation window | | `avg_isl` / `avg_osl` | Mean raw input and output lengths | | `avg_kv_hit_rate` | Mean router prefix-cache hit rate at admission | | `avg_accept_length` | Mean visible output tokens per decode request-forward | KV hit rate and speculative accept length use last-value semantics in the Planner. Missing accept length samples preserve the previous valid value. Without valid speculative-decoding metadata, the effective accept length is `1.0`. Speculative decoding changes the Planner's effective decode latency and capacity calculations. It does not rewrite raw output length, which remains the input for KV residency, context-length, and request-length calculations. The simulation adapter cannot auto-detect the GPU count from a deployment. Planner experiments must set `prefill_engine_num_gpu` and `decode_engine_num_gpu` explicitly when cumulative GPU-hours are part of the analysis. ## Timing models DynoSim can use the default AIConfigurator-backed timing model or explicit fixed and polynomial timing from `engine.workers..timing`. The timing model predicts prefill and decode duration. The Mocker engine still owns batching, KV-cache state, prefix reuse, preemption, and request progression. AIConfigurator compatibility APIs from the `aisimulate` wheel are used in two distinct places: - `engine.workers..timing.type: default` configures the Mocker forward-pass timing model - `router.prefill_load_model.type: aic` configures router-side prompt-load estimation Keeping these paths separate makes it possible to test router estimates independently from engine timing. ## Related documentation - [Mocker Engine Architecture](/dynamo/knowledge-base/modular-components/backends/mocker/mocker-engine-architecture) - [DynoSim Replay CLI Reference](/dynamo/reference/components/dyno-sim-replay-cli-reference) - [DynoSim Sweep Reference](/dynamo/reference/components/dyno-sim-sweep-reference)