Benchmark Control Hooks

AIPerf can POST control-plane requests to your inference server around a benchmark cell: reset the KV cache once before the run, and start/stop a server profiler around each profiling phase (not warmup).

Control traffic uses an isolated HTTP client. It is not recorded as benchmark requests and does not appear in metrics or profile exports.

Quick start

$
aiperf profile \
>
  --model your-model \
>
  --url http://localhost:8000 \
>
  --endpoint-type chat \
>
  --streaming \
>
  --concurrency 8 \
>
  --request-count 100 \
>
  --reset-kv-cache \
>
  --server-profiler

With defaults this issues:

{origin} is scheme://host:port for each configured --url.

YAML: false | true | object

Under endpoint, both hooks accept a bool or an object:

1
benchmark:
2
  models: [your-model]
3
  endpoint:
4
    url: http://localhost:8000
5
    type: chat
6
    # Disabled (default when omitted)
7
    reset_kv_cache: false
8
    server_profiler: false

1
benchmark:
2
  models: [your-model]
3
  endpoint:
4
    url: http://localhost:8000
5
    type: chat
6
    # Enabled with defaults
7
    reset_kv_cache: true
8
    server_profiler: true

1
benchmark:
2
  models: [your-model]
3
  endpoint:
4
    url: http://localhost:8000
5
    type: chat
6
    reset_kv_cache:
7
      path: /v1/admin/reset_prefix_cache
8
      timeout_seconds: 30
9
    server_profiler:
10
      start_path: /v1/admin/start_profile
11
      stop_path: /v1/admin/stop_profile
12
      timeout_seconds: 15

CLI flags

Setting any path/timeout override also enables the corresponding hook (same as passing true / --reset-kv-cache / --server-profiler).

Defaults and relative paths

Paths must be relative (start with /) and must not contain ://. AIPerf joins each path to every endpoint URL origin. Absolute URLs in path / start_path / stop_path are rejected at config validation.

Control hooks require HTTP transport. Non-HTTP transports raise a validation error when either hook is enabled.

Lifecycle and ownership

• Reset runs in the CLI single-run / multi-run path once per logical benchmark cell (each sweep cell, each multi-trial subprocess), before services start issuing load.
• Profiler is owned by TimingManager / PhaseOrchestrator. Workers never fire control hooks.
• Warmup phases never start or stop the profiler.
• The reset is a per-cell isolation boundary, not a cold-cache guarantee for the profiling phase. It fires before the benchmark services start, so if the run also has a warmup phase, warmup traffic repopulates the prefix cache before profiling begins. That is the intended behavior: the hook exists so one sweep cell (or multi-run trial) cannot inherit cache state from the previous one, and warmup exists precisely so profiling measures a steady-state server. If you want profiling to measure genuinely cold-cache behavior, run without a warmup phase at all so the reset is the last thing that touches the server before profiling load starts. Warmup is opt-in: omit --warmup-request-count / --warmup-num-sessions / --warmup-duration on the CLI, or omit the warmup: block (and any exclude_from_results warmup phase) in YAML. Note that --warmup-request-count 0 is not the way to do this — the flag is constrained to > 0 and a 0 fails validation with Input should be greater than 0.
• Multi-URL endpoints: control POSTs go to each unique origin (scheme://host:port); duplicate path-qualified URLs on the same host are deduplicated. A partial profiler-start failure best-effort stops already-started origins, then re-raises. Profiler stop attempts every unique origin and aggregates failures.
• Seamless non-final profiling: when a profiling phase has seamless=True and is not the last phase, profiler start still runs before send begins, but profiler stop is deferred until the phase drain callback (after in-flight credits return), not when run() returns at send-complete.

Failure policy

Auth headers match readiness probes (Authorization: Bearer … or Anthropic x-api-key, plus any --header / YAML endpoint.headers).

Mock server

The in-repo mock server exposes countable admin routes for local verification:

• POST /reset_prefix_cache (vLLM-style)
• POST /flush_cache (SGLang-style alias; same counter)
• POST /start_profile
• POST /stop_profile

These routes are outside the inference auth path and do not contribute to benchmark records.

For per-server path mappings (vLLM, SGLang, TensorRT-LLM), see Control Hooks by Server.