Skip to navigation

aitune.torch.integrations.device

View as MarkdownOpen in Claude

Device reporting for modules containing tuned submodules.

Module Contents

Functions

NameDescription
get_tuned_execution_deviceReturn the execution device of the tuned submodule owning the first parameter or buffer of a module.
patch_device_propertyMake the device property of a library base class report the execution device of tuned submodules.

API

aitune.torch.integrations.device.get_tuned_execution_device(
module: torch.nn.Module
) -> torch.device | None

Return the execution device of the tuned submodule owning the first parameter or buffer of a module.

Parameters:

module
nn.Module

Module whose first parameter or buffer determines its reported device.

Returns: torch.device | None

The device of the nearest AOT- or JIT-tuned module containing that tensor, or None if there is none.

Examples:

>>> get_tuned_execution_device(torch.nn.Linear(2, 2)) is None
True
aitune.torch.integrations.device.patch_device_property(
cls: type
) -> None

Make the device property of a library base class report the execution device of tuned submodules.

Diffusers and Transformers report the device of a module’s first parameter or buffer. After tuning, parameters of tuned submodules are offloaded to meta, so their parents would report the meta device.

Parameters:

cls
type

Class defining a device property. Classes without one are left unchanged.