aitune.torch.integrations.device
Device reporting for modules containing tuned submodules.
Module Contents
Functions
API
Return the execution device of the tuned submodule owning the first parameter or buffer of a module.
Parameters:
Module whose first parameter or buffer determines its reported device.
Returns: torch.device | None
The device of the nearest AOT- or JIT-tuned module containing that tensor, or None if there is none.
Examples:
Make the device property of a library base class report the execution device of tuned submodules.
Diffusers and Transformers report the device of a module’s first parameter or buffer. After tuning, parameters of tuned submodules are offloaded to meta, so their parents would report the meta device.
Parameters:
Class defining a device property. Classes without one are left unchanged.