DynoSim Sweep Reference | NVIDIA Dynamo Documentation

dynamo.profiler.utils.replay_optimize searches simulated deployment candidates by repeatedly calling the DynoSim run harness. This page describes the Python API. For a guided workflow, see Sweep DynoSim Configurations.

Input models

The API takes a ReplayOptimizeSpec with the following models and fields.

EngineSpec

Field	Purpose
`model`	Model identifier used by timing and candidate-shape providers
`backend`	Backend used for the simulated engine
`baseEngineArgs`	Shared JSON-like engine arguments for aggregated simulations
`prefillEngineArgs`	Prefill-stage arguments for disaggregated simulations
`decodeEngineArgs`	Decode-stage arguments for disaggregated simulations

HardwareSpec

Field	Purpose
`system`	GPU system identifier used by the performance model
`totalGpus`	Maximum simulated GPU budget for a candidate

The GPU budget is a search constraint. It is not a requirement for the host that executes the simulation.

WorkloadSpec

WorkloadSpec accepts either a synthetic workload or a trace.

Common synthetic fields include:

isl
osl
requestCount
concurrency
sharedPrefixRatio
numPrefixGroups

Trace workloads use traceFile and can set arrivalSpeedupRatio. Synthetic-only fields are ignored when traceFile is present. Mooncake-style traces use 512 tokens per hash_id; the underlying run_trace_replay(...) API defaults trace_block_size to 512. WorkloadSpec does not currently expose a separate traceBlockSize field.

SLASpec

SLASpec can constrain:

TTFT
ITL
end-to-end latency
p95 variants of the supported latency bounds

Input fields use DynamoGraphDeploymentRequest-style lowerCamelCase names. The result DataFrames use the DynoSim runner metric names described in Result columns.

RouterSpec

Field	Purpose
`mode`	Routing strategy to search; use `both` to compare round-robin and KV routing
`overlapCredits`	Candidate KV-overlap score credits in the range `0.0` to `1.0`
`prefillLoadScales`	Candidate weights for prompt-side load
`baseConfig`	Base KV-router configuration applied to each candidate

Search fields

Field	Purpose
`objective`	Ranking target, such as throughput, mean TTFT, or mean end-to-end latency
`maxParallelEvals`	Maximum candidate evaluations executed concurrently

Search behavior

The optimizer searches tensor-parallel shapes, worker counts, and router settings under the GPU budget. Each state is evaluated by the DynoSim run harness. States that violate any SLA or budget constraint are retained as infeasible results but cannot become best_feasible.

For disaggregated deployments, the search can vary prefill and decode tensor parallelism and worker counts independently. Aggregated searches use one tensor-parallel size and worker count.

The search is heuristic. It prioritizes states near the GPU-budget edge rather than exhaustively evaluating every possible configuration.

Result object

DenseReplayOptimizationResult contains:

Field	Description
`best_feasible`	Highest-ranked visited state that satisfies all SLA and budget constraints
`best_infeasible`	Highest-ranked visited state that violates at least one constraint
`evaluated_df`	DataFrame containing every evaluated state
`feasible_df`	DataFrame containing only feasible states

Result columns

Common DataFrame columns include:

Category	Columns
Topology	`prefill_tp`, `decode_tp`, `prefill_workers`, `decode_workers`
Routing	`router_mode`, `overlap_score_credit`, `prefill_load_scale`
Budget	`total_gpus_used`
Throughput	`output_throughput_tok_s`
Cache behavior	`prefix_cache_reused_ratio`
Latency	`mean_ttft_ms`, `mean_tpot_ms`, `mean_e2e_latency_ms`

total_gpus_used is the simulated footprint of the candidate. It is not the number of physical GPUs allocated to the process running the sweep.