DocumentationThe warmup phase runs before your actual benchmark to prepare the system for steady-state measurement. This guide explains when and how to configure warmup for accurate benchmarking results.
When benchmarking starts, several “cold-start” effects can pollute your measurements:
Cold-start effects include:
Add warmup with a simple request count:
Sample Output (Successful Run):
This sends 50 warmup requests before the 500 profiling requests begin. Warmup metrics are discarded.
You can trigger warmup with count-based or duration-based stopping:
By default, warmup inherits your profiling settings. Override them for different warmup behavior:
Sample Output (Successful Run):
Warmup runs at 20 concurrent requests, then profiling runs at 100.
Sample Output (Successful Run):
Warmup sends at 10 QPS, then profiling runs at 50 QPS.
Sample Output (Successful Run):
Warmup uses predictable constant arrivals; profiling uses gamma arrivals with reduced variance (smoothness > 1 = smoother than Poisson).
Warmup can include its own gradual ramp-up:
Sample Output (Successful Run):
Timeline:
By default, AIPerf waits indefinitely for all warmup responses before starting profiling. When using duration-based warmup (--warmup-duration), you can limit this wait:
This prevents slow warmup responses from delaying the profiling phase indefinitely.
For multi-turn benchmarks, warmup by session count ensures complete conversations:
This completes 10 full conversations (each ~5 turns) before profiling begins.
When using prefill concurrency to limit simultaneous prefill operations, you can configure warmup separately:
Warmup runs with lower limits (20 concurrent, 2 prefill), then profiling uses full limits.
Just warm up connections and caches:
Simulate gradual traffic increase:
For long prompts, use lower warmup concurrency to avoid OOM:
