Skip to navigation

Task module.

Subpackages

Submodules

Package Contents

Classes

NameDescription
MeasuringStopStrategyStrategy when to stop collecting measurements for given batch size.
MeasuringStrategyStrategy how do the measurement and create a ProfilingResultEvent(s).
ModelExecutionTimeMeasuringStrategyStrategy to measure execution time.
NumStepsMeasuringStopStrategyStrategy to stop collecting measurements after a given number of steps.
ProfilingConfigConfiguration for profiling.
ProfilingResultEventProfiling result event.
ProfilingResultsProfiling results.
ProfilingStatusProfiling status.
StableWindowMeasuringStopStrategyStrategy to stop collecting measurements after a stable sliding window has been collected.

Functions

NameDescription
get_inference_eventsGet inference events.
profileProfile a callable with given dataset.

API

class aitune.torch.task.MeasuringStopStrategy()
Abstract

Strategy when to stop collecting measurements for given batch size.

aitune.torch.task.MeasuringStopStrategy.get_events(
abstract

Get events to use for measuring stop strategy.

aitune.torch.task.MeasuringStopStrategy.should_stop(
) -> bool
abstract

Check if the measurement should be stopped.

Parameters:

results
list[ProfilingResultEvent]

list[ ProfilingResultEvent ] — List of profiling results events for a one measurement.

Returns: bool

True if the measurement should be stopped, False otherwise.

class aitune.torch.task.MeasuringStrategy()
Abstract

Strategy how do the measurement and create a ProfilingResultEvent(s).

aitune.torch.task.MeasuringStrategy.do_measurement(
batch_size: int,
model: collections.abc.Callable,
sample: tuple[list, dict],
kwargs = {}
abstract

Do the measurement and create a ProfilingResultEvent(s).

Parameters:

batch_size
int

Batch size of the measurement.

model
Callable

Model to measure.

sample
tuple[list, dict]

Sample to measure.

**kwargs
Defaults to {}

Additional keyword arguments.

Returns: list[ ProfilingResultEvent ]

List of ProfilingResultEvent(s).

class aitune.torch.task.ModelExecutionTimeMeasuringStrategy()

Bases: MeasuringStrategy

Strategy to measure execution time.

counter
int = 0
aitune.torch.task.ModelExecutionTimeMeasuringStrategy.do_measurement(
batch_size: int,
model: collections.abc.Callable,
sample: tuple[list, dict],
measurement_kwargs = {}

Do the measurement and create a ProfilingResultEvent(s) for the model.

class aitune.torch.task.NumStepsMeasuringStopStrategy(
num_steps: int = 20,
warmup_samples: int = 10
)

Bases: MeasuringStopStrategy

Strategy to stop collecting measurements after a given number of steps.

_min_samples_seen
= self.warmup_samples + self.num_steps
_samples_seen
= 0
aitune.torch.task.NumStepsMeasuringStopStrategy.get_events(

Get final measured events after should_stop() has returned True.

aitune.torch.task.NumStepsMeasuringStopStrategy.should_stop(
) -> bool

Check if the measurement should be stopped.

class aitune.torch.task.ProfilingConfig(
batch_sizes: list[int] | collections.abc.Generator[int, None, None],
batching: bool = True,
)
Dataclass

Configuration for profiling.

Note: all profiling configuration strategies are deepcopied before use.

batch_sizes
list[int] | Generator[int, None, None]

List of batch sizes to profile.

batching
bool = True

Whether profiling should batch samples together.

measurement_stop_strategy
MeasuringStopStrategy | None = None

Strategy to stop collecting measurements.

measuring_strategy
MeasuringStrategy | None = None

Strategy to measure the model.

profiling_stop_strategy
ProfilingStopStrategy | None = None

Strategy to stop profiling.

aitune.torch.task.ProfilingConfig.__post_init__()

Post-init.

class aitune.torch.task.ProfilingResultEvent(
timestamp: float,
model_name: str,
backend_details: str,
batch_size: int,
phase: str,
measurement_id: int = 0,
execution_time: float = 0.0
)
Dataclass

Profiling result event.

backend_details
str

Details of the backend.

batch_size
int

Batch size of the measurement.

execution_time
float = 0.0

Execution time of the measurement.

measurement_id
int = 0

Unique identifier for the measurement.

model_name
str
phase
str

Phase of the measurement.

timestamp
float

Timestamp of the measurement.

class aitune.torch.task.ProfilingResults(
)
Dataclass

Profiling results.

entries
list[ProfilingResultEvent] = field(default_factory=list)
class aitune.torch.task.ProfilingStatus(
status: aitune.torch.task.profiling.profiling.ProfilingStatus.Status,
error: typing.Any | None = None,
)
Dataclass

Profiling status.

error
Any | None = None
results
ProfilingResults | None = None
status
Status
class aitune.torch.task.StableWindowMeasuringStopStrategy(
window_size: int = 20,
max_cv_ratio: float = 0.1,
warmup_samples: int = 10,
max_samples: int = 100
)

Bases: MeasuringStopStrategy

Strategy to stop collecting measurements after a stable sliding window has been collected.

_max_samples_seen
= self.max_samples + self.warmup_samples
_min_samples_seen
= self.window_size + self.warmup_samples
_samples_seen
= 0
_window
list[ProfilingResultEvent] = []
aitune.torch.task.StableWindowMeasuringStopStrategy.get_events(

Get events to use for measuring stop strategy.

aitune.torch.task.StableWindowMeasuringStopStrategy.should_stop(
) -> bool

Check if the measurement should be stopped.

Get inference events.

aitune.torch.task.profile(
func: collections.abc.Callable,
model_name: str | None = None,
backend_details: str | None = None

Profile a callable with given dataset.

Parameters:

func
Callable

Function to profile.

dataset
DatasetLike | DataLoaderFactory | torch.Tensor

DatasetLike | DataLoaderFactory | torch.Tensor — Dataset to profile.

profiling_config
ProfilingConfig

ProfilingConfig — Configuration for profiling.

model_name
str | NoneDefaults to None

Name of the model, optional.

backend_details
str | NoneDefaults to None

Detailed information of the backend, optional.

Returns: ProfilingStatus

Status of the profiling.