aitune.torch.tune_strategy.min_latency_strategy

Min latency tune strategy.

Module Contents

Classes

Name	Description
`MinLatencyProfilingResult`	Profiling result for min-latency selection.
`MinLatencyStrategy`	Searches and selects the backend with minimum latency at batch size 1.

API

class aitune.torch.tune_strategy.min_latency_strategy.MinLatencyProfilingResult(
    selected_batch_size: int = 1,
    latency: float
)

Dataclass

Bases: BackendProfilingResult

Profiling result for min-latency selection.

latency

float

metric

float

Returns latency as the comparison metric.

selected_batch_size

int = 1

class aitune.torch.tune_strategy.min_latency_strategy.MinLatencyStrategy(
    args = (),
    kwargs = {}
)

Bases: ProfilingTuneStrategy

Searches and selects the backend with minimum latency at batch size 1.

TorchEager is profiled in _pre_tune as a latency baseline when baseline validation is enabled (not injected into the backends list). When validation is enabled, the strategy falls back to TorchEager when no user-provided backend beats it. When disabled, the best user-provided backend wins and the strategy raises if all user backends fail.

_description

_metric_label

= 'latency'

_metric_unit

= 'ms'

_title

= 'Min Latency Strategy'

_value_fmt

= '.3f'

aitune.torch.tune_strategy.min_latency_strategy.MinLatencyStrategy._get_profiling_config(
    batching: bool,
    max_batch_size: int
) -> aitune.torch.task.profiling.ProfilingConfig

aitune.torch.tune_strategy.min_latency_strategy.MinLatencyStrategy._is_better(
    result: aitune.torch.tune_strategy.profiling_tune_strategy.BackendProfilingResult,
    other: aitune.torch.tune_strategy.profiling_tune_strategy.BackendProfilingResult
) -> bool

aitune.torch.tune_strategy.min_latency_strategy.MinLatencyStrategy._measure(
    backend: aitune.torch.backend.Backend,
    name: str,
    graph_spec: aitune.torch.module.graph_spec.GraphSpec,
    data: list[aitune.torch.module.recording_module.Sample],
    profiling_cfg: aitune.torch.task.profiling.ProfilingConfig
) -> aitune.torch.tune_strategy.min_latency_strategy.MinLatencyProfilingResult

Profiles the backend at batch size 1 and returns latency.

aitune.torch.tune_strategy.min_latency_strategy.MinLatencyStrategy._speedup(
    result: aitune.torch.tune_strategy.profiling_tune_strategy.BackendProfilingResult,
    baseline_result: aitune.torch.tune_strategy.profiling_tune_strategy.BackendProfilingResult
) -> float