> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# DynoSim Sweep Reference

`dynamo.profiler.utils.replay_optimize` searches simulated deployment candidates by repeatedly
calling the DynoSim run harness. This page describes the Python API. For a guided workflow, see
[Sweep DynoSim Configurations](/dynamo/dev/kubernetes/operations/dynosim/simulation-sweeps).

## Input models

The API takes a `ReplayOptimizeSpec` with the following models and fields.

### EngineSpec

| Field               | Purpose                                                       |
| ------------------- | ------------------------------------------------------------- |
| `model`             | Model identifier used by timing and candidate-shape providers |
| `backend`           | Backend used for the simulated engine                         |
| `baseEngineArgs`    | Shared JSON-like engine arguments for aggregated simulations  |
| `prefillEngineArgs` | Prefill-stage arguments for disaggregated simulations         |
| `decodeEngineArgs`  | Decode-stage arguments for disaggregated simulations          |

### HardwareSpec

| Field       | Purpose                                             |
| ----------- | --------------------------------------------------- |
| `system`    | GPU system identifier used by the performance model |
| `totalGpus` | Maximum simulated GPU budget for a candidate        |

The GPU budget is a search constraint. It is not a requirement for the host that executes the
simulation.

### WorkloadSpec

`WorkloadSpec` accepts either a synthetic workload or a trace.

Common synthetic fields include:

* `isl`
* `osl`
* `requestCount`
* `concurrency`
* `sharedPrefixRatio`
* `numPrefixGroups`

Trace workloads use `traceFile` and can set `arrivalSpeedupRatio`. Synthetic-only fields are ignored
when `traceFile` is present. Mooncake-style traces use 512 tokens per `hash_id`; the underlying
`run_trace_replay(...)` API defaults `trace_block_size` to `512`. `WorkloadSpec` does not currently
expose a separate `traceBlockSize` field.

### SLASpec

`SLASpec` can constrain:

* TTFT
* ITL
* end-to-end latency
* p95 variants of the supported latency bounds

Input fields use DynamoGraphDeploymentRequest-style lowerCamelCase names. The result DataFrames use
the DynoSim runner metric names described in [Result columns](#result-columns).

### RouterSpec

| Field               | Purpose                                                                      |
| ------------------- | ---------------------------------------------------------------------------- |
| `mode`              | Routing strategy to search; use `both` to compare round-robin and KV routing |
| `overlapCredits`    | Candidate KV-overlap score credits in the range `0.0` to `1.0`               |
| `prefillLoadScales` | Candidate weights for prompt-side load                                       |
| `baseConfig`        | Base KV-router configuration applied to each candidate                       |

### Search fields

| Field              | Purpose                                                                   |
| ------------------ | ------------------------------------------------------------------------- |
| `objective`        | Ranking target, such as throughput, mean TTFT, or mean end-to-end latency |
| `maxParallelEvals` | Maximum candidate evaluations executed concurrently                       |

## Search behavior

The optimizer searches tensor-parallel shapes, worker counts, and router settings under the GPU
budget. Each state is evaluated by the DynoSim run harness. States that violate any SLA or budget
constraint are retained as infeasible results but cannot become `best_feasible`.

For disaggregated deployments, the search can vary prefill and decode tensor parallelism and worker
counts independently. Aggregated searches use one tensor-parallel size and worker count.

The search is heuristic. It prioritizes states near the GPU-budget edge rather than exhaustively
evaluating every possible configuration.

## Result object

`DenseReplayOptimizationResult` contains:

| Field             | Description                                                                |
| ----------------- | -------------------------------------------------------------------------- |
| `best_feasible`   | Highest-ranked visited state that satisfies all SLA and budget constraints |
| `best_infeasible` | Highest-ranked visited state that violates at least one constraint         |
| `evaluated_df`    | DataFrame containing every evaluated state                                 |
| `feasible_df`     | DataFrame containing only feasible states                                  |

## Result columns

Common DataFrame columns include:

| Category       | Columns                                                        |
| -------------- | -------------------------------------------------------------- |
| Topology       | `prefill_tp`, `decode_tp`, `prefill_workers`, `decode_workers` |
| Routing        | `router_mode`, `overlap_score_credit`, `prefill_load_scale`    |
| Budget         | `total_gpus_used`                                              |
| Throughput     | `output_throughput_tok_s`                                      |
| Cache behavior | `prefix_cache_reused_ratio`                                    |
| Latency        | `mean_ttft_ms`, `mean_tpot_ms`, `mean_e2e_latency_ms`          |

`total_gpus_used` is the simulated footprint of the candidate. It is not the number of physical GPUs
allocated to the process running the sweep.

## Related pages

* [Run a DynoSim Simulation](/dynamo/dev/kubernetes/operations/dynosim/simulation-runs)
* [DynoSim Replay CLI Reference](/dynamo/dev/components/dyno-sim-replay-cli-reference)
* [DynoSim Architecture](/dynamo/dev/knowledge-base/design-documents/dyno-sim-architecture)