> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/aiperf/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/aiperf/_mcp/server.

# Adaptive Scale

Adaptive scale is a single-run load controller for finding and sustaining an SLA boundary. Instead of launching many independent sweep or search runs, AIPerf starts at a low control value, evaluates SLA windows, increases load while every SLA filter passes, and then sustains near the last passing boundary.

Use adaptive scale when you want one benchmark invocation to push a service until a latency or throughput constraint starts failing, then keep pressure near that edge. Use `adaptive_search` when you want offline Bayesian optimization across multiple runs, sweeps when you want a fixed experiment grid, fixed ramps when you already know the schedule you want to replay, and static concurrency or request rate when you only need one fixed load point.

## YAML example

YAML is the preferred way to configure adaptive scale because it keeps the control variable, assessment windows, sustain period, and SLA filters together. The canonical shape uses a nested `adaptive_scale.control` block:

```yaml
schemaVersion: "2.0"

benchmark:
  model: meta-llama/Llama-3.1-8B-Instruct
  endpoint:
    url: http://localhost:8000/v1/chat/completions
    type: chat
    streaming: true
  dataset:
    type: synthetic
    entries: 1000
    prompts: {isl: 512, osl: 128}
  phases:
    - name: profiling
      type: concurrency
      concurrency: 200
      prefill_concurrency: 64
      duration: 3600
      adaptive_scale:
        enabled: true
        control:
          variable: prefill_concurrency
          min: 1
          max: 64
        assessment_period: 60
        min_completed_requests: 20
        sustain_duration: 1800
        strategy:
          type: ramp_until_fail
          step_policy: sla_margin
          base_step: 10
          max_step_multiplier: 4
      sla:
        request_latency:
          p95:
            le: 30000
        request_throughput:
          avg:
            ge: 1
```

A window passes only when every SLA filter passes. Latency thresholds use milliseconds. Lower-is-better metrics usually use `lt` or `le`; higher-is-better metrics such as request throughput and goodput ratio usually use `gt` or `ge`.

The canonical adaptive shape uses `adaptive_scale.control` for the load-control variable and phase-level `sla` for the filters. Existing flat concurrency aliases such as `min_concurrency` and `max_concurrency` are still accepted for compatibility, but new configs should use `control.min` and `control.max`.

Do not combine adaptive scale with a fixed ramp on the same variable. For example, `control.variable: prefill_concurrency` cannot be used with `prefill_ramp`, and `control.variable: request_rate` cannot be used with `rate_ramp`.

## Boundary Example

This example demonstrates adaptive scale discovering a latency boundary against an OpenAI-compatible chat endpoint. Make sure the endpoint is reachable at `localhost:8000`, then tune the `model` and SLA threshold for your service.

```yaml
schemaVersion: "2.0"

benchmark:
  model: Qwen/Qwen3-0.6B
  endpoint:
    url: http://localhost:8000/v1/chat/completions
    type: chat
    streaming: true
  dataset:
    type: synthetic
    entries: 1000
    prompts:
      isl: 128
      osl: 16
  phases:
    - name: profiling
      type: concurrency
      concurrency: 4
      duration: 45
      adaptive_scale:
        enabled: true
        control:
          variable: concurrency
          min: 4
          max: 64
        assessment_period: 5
        min_completed_requests: 5
        sustain_duration: 10
        strategy:
          type: ramp_until_fail
          step_policy: fixed_percent_step
          step_percent: 100
      sla:
        request_latency:
          p95:
            le: 80
```

Run it with:

```bash
aiperf profile \
  --config adaptive-scale-boundary.yaml \
  --tokenizer builtin \
  --extra-inputs ignore_eos:true \
  --ui none \
  --output-artifact-dir ./adaptive-scale-boundary-artifacts
```

**Sample Output (Successful Run):**

```text
concurrency 4  -> request_latency:p95 = 54.1 ms, pass
concurrency 8  -> request_latency:p95 = 58.7 ms, pass
concurrency 16 -> request_latency:p95 = 56.1 ms, pass
concurrency 32 -> request_latency:p95 = 73.7 ms, pass
concurrency 64 -> request_latency:p95 = 98.2 ms, fail
sustain at 32  -> request_latency:p95 = 72.0 ms, then 74.4 ms, pass
```

The corresponding `adaptive_scale_summary.json` included:

```json
{
  "status": "completed",
  "completed_reason": "sustain_duration_completed",
  "control_variable": "concurrency",
  "control_value": 32,
  "boundary_value": 32,
  "last_passing_value": 32,
  "first_failing_value": 64,
  "sustain_windows": 2,
  "sustain_passed_windows": 2
}
```

Exact numbers depend on your server and hardware. If the run passes at `max`, lower the latency threshold or raise `adaptive_scale.control.max`. If it fails at `min`, loosen the latency threshold or lower `adaptive_scale.control.min`.

## Control variables

| Control variable | Phase shape | What changes |
| --- | --- | --- |
| `concurrency` | `type: concurrency` or another phase with a concurrency ceiling | In-flight session concurrency. |
| `prefill_concurrency` | Streaming phases with `concurrency` and `prefill_concurrency` | Simultaneous prefill-heavy requests, while total concurrency remains capped. |
| `request_rate` | Rate-controlled phases such as `poisson`, `constant`, or `gamma` | Scheduled request rate. `concurrency` still acts as an in-flight ceiling when set. |
| `users` | `type: user_centric` | Live simulated user timelines. Total target QPS remains fixed, so per-user turn gap changes. |

For `users`, adaptive scale changes population pressure rather than acting as another spelling of request rate.

## CLI quick start

The CLI is useful for scripts and simple one-phase runs. Prefer YAML for reusable benchmark definitions.

```bash
aiperf profile \
  --url http://localhost:8000/v1/chat/completions \
  --model meta-llama/Llama-3.1-8B-Instruct \
  --endpoint-type chat \
  --streaming \
  --concurrency 400 \
  --benchmark-duration 3600 \
  --adaptive-scale \
  --adaptive-scale-control concurrency:1,400:int \
  --adaptive-scale-assessment-period 60 \
  --adaptive-sustain-duration 1800 \
  --adaptive-scale-sla request_latency:p95:le:30000
```

The compact control form is `--adaptive-scale-control variable:min,max:type`. Use `int` for `concurrency`, `prefill_concurrency`, and `users`; use `float` for `request_rate`. Do not mix compact control with expanded `--adaptive-control-*` flags.

## Artifacts

Adaptive scale writes these timing-owned artifacts into the run artifact directory:

```text
adaptive_scale_events.jsonl
adaptive_scale_summary.json
```

`adaptive_scale_events.jsonl` is an event stream for orchestration and post-processing. Each line includes `schema_version`, timestamps, `event`, `control_variable`, `control_value_before`, `control_value_after`, `boundary_value`, `last_passing_value`, `first_failing_value`, `sla_values`, and `binding_sla` fields. Pollers should key off explicit events such as `sustain_started` rather than sleeping for a fixed amount of time.

`adaptive_scale_summary.json` is the final controller summary. It records the discovered boundary, final control value, last passing value, first failing value, sustain status, throughput, sample counts, error counts, cancellation counts, and the evaluated candidate windows.

Artifact fields are intended for orchestration-facing consumers, so treat schema changes and field renames as compatibility events.

## Metric semantics

Use `request_latency` for simple latency-boundary examples. Request throughput aliases and `goodput_ratio` can also be used as adaptive SLA filters. See [Adaptive SLA metric support](/aiperf/dev/tutorials/configuration/yaml-configuration-files#adaptive-sla-metric-support) for the full metric and statistic matrix.