Skip to navigation

aitune.torch.tune_strategy.resolver

View as Markdown

Create tune strategies whose backends are resolved for each module.

Module Contents

Classes

NameDescription
ConstraintDescribe one requirement that a tuning candidate must satisfy.
DynamicTuneStrategyDescribes rules that are used to select and configure an optimal tuning strategy for a given module at tune time.

Functions

NameDescription
_resolve_backendsCreate default candidates compatible with the module and requested compilation.
materialize_strategyResolve a dynamic strategy for module or preserve an explicit strategy.
resolve_strategyDeclare a strategy whose backends will be resolved for each tuned module.

Data

Compilation

Objective

Relation

StrategyOption

Unit

_COMPILATION_MODES

_OBJECTIVES

API

class aitune.torch.Constraint()

Bases: BaseModel

Describe one requirement that a tuning candidate must satisfy.

metric
Objective
model_config
= ConfigDict(frozen=True, extra='forbid')
relation
Relation
unit
Unit = 'ms'
value
PositiveFloat
aitune.torch.Constraint._metric_and_unit_validation() -> typing.Self

Only ‘latency’ metric with ‘ms’ unit is supported for now.

aitune.torch.Constraint.max_latency_ms(
milliseconds: float
) -> typing.Self
classmethod

Require latency to be no greater than milliseconds.

class aitune.torch.tune_strategy.resolver.DynamicTuneStrategy()

Bases: BaseModel

Describes rules that are used to select and configure an optimal tuning strategy for a given module at tune time.

This class holds no backend instances or backend-specific state. Instead, it contains a declarative description of the desired optimization objective (such as “throughput” or “latency”) and a set of constraints, all of which apply together.

In this framework:

  • The objective defines what we try to maximize or minimize (e.g., maximize throughput).
  • The constraint(s) express requirements that must be met by any configuration the strategy chooses (such as “latency at_most N ms”). For now, only a single latency constraint is supported (e.g., “latency at_most N ms”).
  • Typically, this means we optimize for maximum throughput, but only among strategies/configurations that meet the given latency constraint.

This class is dynamic: it does not directly reference any backends. When materialize() is called with a particular module, it creates a concrete TuneStrategy with appropriate backend objects and automatically selects the right strategy class (e.g., LatencyBudgetStrategy for a throughput-optimization subject to a latency constraint).

The same DynamicTuneStrategy instance can be used for multiple modules, and a fresh concrete strategy is produced for each module.

Example:

Optimize for max throughput, but only if latency ≤ 40 ms for the current module

dts = DynamicTuneStrategy( objective=“throughput”, compilation=“any”, constraints=[Constraint.max_latency_ms(40)] ) strategy = dts.materialize(some_module)

_find_max_batch_size
bool | None = None
compilation
Compilation = 'any'
constraints
Sequence[Constraint] = Field(default_factory=list)
objective
Objective
aitune.torch.tune_strategy.resolver.DynamicTuneStrategy._validate_constraints(
v
)
classmethod
aitune.torch.tune_strategy.resolver.DynamicTuneStrategy.clone() -> typing.Self

Return an independent copy of this dynamic configuration.

aitune.torch.tune_strategy.resolver.DynamicTuneStrategy.enable_find_max_batch_size(
enable: bool = True
) -> typing.Self

Configure maximum-batch-size discovery on the resolved strategy.

aitune.torch.tune_strategy.resolver.DynamicTuneStrategy.materialize(
module: torch.nn.Module

Create a concrete strategy with fresh backends for module.

aitune.torch.tune_strategy.resolver.DynamicTuneStrategy.to_json_dict() -> dict

Return the unresolved configuration for reporting.

aitune.torch.tune_strategy.resolver._resolve_backends(
module: torch.nn.Module,

Create default candidates compatible with the module and requested compilation.

aitune.torch.tune_strategy.resolver.materialize_strategy(
module: torch.nn.Module

Resolve a dynamic strategy for module or preserve an explicit strategy.

aitune.torch.resolve_strategy(
constraints: collections.abc.Sequence[aitune.torch.tune_strategy.resolver.Constraint] = ()

Declare a strategy whose backends will be resolved for each tuned module.

Parameters:

objective
ObjectiveDefaults to 'throughput'

Objective — Performance metric to optimize: throughput or latency.

compilation
CompilationDefaults to 'any'

Compilation — Allow ahead-of-time, just-in-time, or any supported backend build mode.

constraints
Sequence[Constraint]Defaults to ()

Sequence[ Constraint ] — Limits that candidates must satisfy while optimizing the objective.

Returns: DynamicTuneStrategy

A dynamic strategy configuration suitable for AOT and JIT tuning.

aitune.torch.tune_strategy.resolver.Compilation = Literal['aot', 'jit', 'any']
aitune.torch.tune_strategy.resolver.Objective = Literal['throughput', 'latency']
aitune.torch.tune_strategy.resolver.Relation = Literal['at_most']
aitune.torch.tune_strategy.resolver.StrategyOption = TuneStrategy | DynamicTuneStrategy
aitune.torch.tune_strategy.resolver.Unit = Literal['ms']
aitune.torch.tune_strategy.resolver._COMPILATION_MODES = frozenset({'aot', 'jit', 'any'})
aitune.torch.tune_strategy.resolver._OBJECTIVES = frozenset({'throughput', 'latency'})