Skip to navigation

aitune.torch.task.profiling

View as MarkdownOpen in Claude

Submodules

Package Contents

Classes

NameDescription
AllSamplesProfilingStopStrategyStrategy to do not stop profiling until all samples are processed.
MeasuringStopStrategyStrategy when to stop collecting measurements for given batch size.
MeasuringStrategyStrategy how do the measurement and create a ProfilingResultEvent(s).
ModelExecutionTimeMeasuringStrategyStrategy to measure execution time.
NumStepsMeasuringStopStrategyStrategy to stop collecting measurements after a given number of steps.
ProfilingConfigConfiguration for profiling.
ProfilingResultEventProfiling result event.
ProfilingResultsProfiling results.
ProfilingStatusProfiling status.
ProfilingStopStrategyStrategy when to stop profiling.
StableWindowMeasuringStopStrategyStrategy to stop collecting measurements after a stable sliding window has been collected.
ThroughputSaturatedProfilingStopStrategyStrategy to stop profiling when throughput is saturated.

Functions

NameDescription
get_throughputGet throughput from profiling results.
is_throughput_saturatedValidate if throughput saturated between consecutive samples.
profileProfile a callable with given dataset.

API

class aitune.torch.task.profiling.AllSamplesProfilingStopStrategy()
Dataclass

Bases: ProfilingStopStrategy

Strategy to do not stop profiling until all samples are processed.

aitune.torch.task.profiling.AllSamplesProfilingStopStrategy.should_stop(
) -> bool

Check if the profiling should be stopped.

class aitune.torch.task.MeasuringStopStrategy()
Abstract

Strategy when to stop collecting measurements for given batch size.

aitune.torch.task.MeasuringStopStrategy.get_events(
abstract

Get events to use for measuring stop strategy.

aitune.torch.task.MeasuringStopStrategy.should_stop(
) -> bool
abstract

Check if the measurement should be stopped.

Parameters:

results
list[ProfilingResultEvent]

list[ ProfilingResultEvent ] — List of profiling results events for a one measurement.

Returns: bool

True if the measurement should be stopped, False otherwise.

class aitune.torch.task.MeasuringStrategy()
Abstract

Strategy how do the measurement and create a ProfilingResultEvent(s).

aitune.torch.task.MeasuringStrategy.do_measurement(
batch_size: int,
model: collections.abc.Callable,
sample: tuple[list, dict],
kwargs = {}
abstract

Do the measurement and create a ProfilingResultEvent(s).

Parameters:

batch_size
int

Batch size of the measurement.

model
Callable

Model to measure.

sample
tuple[list, dict]

Sample to measure.

**kwargs
Defaults to {}

Additional keyword arguments.

Returns: list[ ProfilingResultEvent ]

List of ProfilingResultEvent(s).

class aitune.torch.task.ModelExecutionTimeMeasuringStrategy()

Bases: MeasuringStrategy

Strategy to measure execution time.

counter
int = 0
aitune.torch.task.ModelExecutionTimeMeasuringStrategy.do_measurement(
batch_size: int,
model: collections.abc.Callable,
sample: tuple[list, dict],
measurement_kwargs = {}

Do the measurement and create a ProfilingResultEvent(s) for the model.

class aitune.torch.task.NumStepsMeasuringStopStrategy(
num_steps: int = 20,
warmup_samples: int = 10
)

Bases: MeasuringStopStrategy

Strategy to stop collecting measurements after a given number of steps.

_min_samples_seen
= self.warmup_samples + self.num_steps
_samples_seen
= 0
aitune.torch.task.NumStepsMeasuringStopStrategy.get_events(

Get final measured events after should_stop() has returned True.

aitune.torch.task.NumStepsMeasuringStopStrategy.should_stop(
) -> bool

Check if the measurement should be stopped.

class aitune.torch.task.ProfilingConfig(
batch_sizes: list[int] | collections.abc.Generator[int, None, None],
batching: bool = True,
)
Dataclass

Configuration for profiling.

Note: all profiling configuration strategies are deepcopied before use.

batch_sizes
list[int] | Generator[int, None, None]

List of batch sizes to profile.

batching
bool = True

Whether profiling should batch samples together.

measurement_stop_strategy
MeasuringStopStrategy | None = None

Strategy to stop collecting measurements.

measuring_strategy
MeasuringStrategy | None = None

Strategy to measure the model.

profiling_stop_strategy
ProfilingStopStrategy | None = None

Strategy to stop profiling.

aitune.torch.task.ProfilingConfig.__post_init__()

Post-init.

class aitune.torch.task.ProfilingResultEvent(
timestamp: float,
model_name: str,
backend_details: str,
batch_size: int,
phase: str,
measurement_id: int = 0,
execution_time: float = 0.0
)
Dataclass

Profiling result event.

backend_details
str

Details of the backend.

batch_size
int

Batch size of the measurement.

execution_time
float = 0.0

Execution time of the measurement.

measurement_id
int = 0

Unique identifier for the measurement.

model_name
str
phase
str

Phase of the measurement.

timestamp
float

Timestamp of the measurement.

class aitune.torch.task.ProfilingResults(
)
Dataclass

Profiling results.

entries
list[ProfilingResultEvent] = field(default_factory=list)
class aitune.torch.task.ProfilingStatus(
status: aitune.torch.task.profiling.profiling.ProfilingStatus.Status,
error: typing.Any | None = None,
)
Dataclass

Profiling status.

error
Any | None = None
results
ProfilingResults | None = None
status
Status
class aitune.torch.task.profiling.ProfilingStopStrategy()
Abstract

Strategy when to stop profiling.

aitune.torch.task.profiling.ProfilingStopStrategy.should_stop(
) -> bool
abstract

Check if the profiling should be stopped.

Parameters:

results
list[ProfilingResultEvent]

list[ ProfilingResultEvent ] — List of profiling results.

Returns: bool

True if the profiling should be stopped, False otherwise.

class aitune.torch.task.StableWindowMeasuringStopStrategy(
window_size: int = 20,
max_cv_ratio: float = 0.1,
warmup_samples: int = 10,
max_samples: int = 100
)

Bases: MeasuringStopStrategy

Strategy to stop collecting measurements after a stable sliding window has been collected.

_max_samples_seen
= self.max_samples + self.warmup_samples
_min_samples_seen
= self.window_size + self.warmup_samples
_samples_seen
= 0
_window
list[ProfilingResultEvent] = []
aitune.torch.task.StableWindowMeasuringStopStrategy.get_events(

Get events to use for measuring stop strategy.

aitune.torch.task.StableWindowMeasuringStopStrategy.should_stop(
) -> bool

Check if the measurement should be stopped.

class aitune.torch.task.profiling.ThroughputSaturatedProfilingStopStrategy(
min_throughput_gain_ratio: float = 0.05,
throughput_backoff_limit: int = 2,
_backoff_counter: int = 0
)
Dataclass

Bases: ProfilingStopStrategy

Strategy to stop profiling when throughput is saturated.

Additionally, backoff policy is introduced, allowing for few more measurements after saturation is detected. This is useful to avoid stopping profiling too early, when the model is still not fully optimized.

Parameters:

min_throughput_gain_ratio
floatDefaults to 0.05

Minimum relative throughput gain required to continue profiling.

throughput_backoff_limit
intDefaults to 2

Number of measurements after saturation is detected to continue profiling. 0 means no backoff.

_backoff_counter
int = 0
_best_results
list[ProfilingResultEvent] = field(default_factory=list)
min_throughput_gain_ratio
float = 0.05
throughput_backoff_limit
int = 2
aitune.torch.task.profiling.ThroughputSaturatedProfilingStopStrategy.__post_init__()

Validate ratio configuration.

aitune.torch.task.profiling.ThroughputSaturatedProfilingStopStrategy.should_stop(
) -> bool

Check if the profiling should be stopped.

aitune.torch.task.profiling.get_throughput(
batch_size: int | None = None
) -> float

Get throughput from profiling results.

Parameters:

results
list[ProfilingResultEvent]

list[ ProfilingResultEvent ] — List of profiling results.

batch_size
int | NoneDefaults to None

Batch size to use for throughput calculation. If not provided, it will be inferred from the results.

Returns: float

Throughput in samples per second.

aitune.torch.task.profiling.is_throughput_saturated(
min_throughput_gain_ratio: float,
) -> bool

Validate if throughput saturated between consecutive samples.

Parameters:

profiling_result
list[ProfilingResultEvent]

list[ ProfilingResultEvent ] — Current profiling results.

min_throughput_gain_ratio
float

Minimum relative throughput gain required to continue profiling.

prev_results
list[ProfilingResultEvent]

list[ ProfilingResultEvent ] — Previous profiling results or previous throughput.

Returns: bool

True when throughput saturated between consecutive samples.

aitune.torch.task.profile(
func: collections.abc.Callable,
model_name: str | None = None,
backend_details: str | None = None

Profile a callable with given dataset.

Parameters:

func
Callable

Function to profile.

dataset
DatasetLike | DataLoaderFactory | torch.Tensor

DatasetLike | DataLoaderFactory | torch.Tensor — Dataset to profile.

profiling_config
ProfilingConfig

ProfilingConfig — Configuration for profiling.

model_name
str | NoneDefaults to None

Name of the model, optional.

backend_details
str | NoneDefaults to None

Detailed information of the backend, optional.

Returns: ProfilingStatus

Status of the profiling.