Sweeper Tutorial

1. Define the Backend Search

Choose a model, hardware system, deployment modes, backends, and GPU budget:

search_space:
  model_name: meta-llama/Meta-Llama-3.1-8B
  hardware_sku: gb200
  deployment_mode: [agg]
  backend: [trtllm]
  gpu_budget: 8

Sweeper enumerates legal parallel configurations, removes unsupported runner topologies, and adds the active engine-role knobs to the optimizer study.

2. Define One Workload and Goal

workload:
  isl: 1024
  osl: 128
  request_rate: 4
  num_request_ratio: 10
goal:
  target: throughput

Every candidate is evaluated against this workload. See Traffic for trace and closed-loop alternatives, and Optimization Goals for SLA and Pareto modes.

3. Control the Sweep

sweep:
  max_rounds: 2
  candidates_per_round: 4
  parallel_evals: 2
  max_eval_seconds: 300

Each round is a barrier: the sampler asks for unique candidates, replay evaluates them, and then the scores are reported back together. parallel_evals controls replay worker fan-out.

4. Add Optional Feature Search

An installed or injected provider owns the schema below its adapter name:

adapters:
  example.policy:
    search_space:
      mode: [balanced, latency]

The provider receives the complete search_space mapping. It does not receive a preselected concrete feature configuration.

5. Run and Inspect

sweeper = Sweeper(
    runner_factory=my_runner_factory,
    providers={"example.policy": my_provider},
)
candidates = sweeper.run(config)
for candidate in candidates[:5]:
    print(candidate.score, candidate.used_gpus, candidate.config)

The same Sweeper instance can run multiple configurations. Studies, caches, and process pools are new for every call.