Skip to navigation

aitune.torch.call_tree

View as MarkdownOpen in Claude

Call tree: which recorded calls ran inside which, keyed by their chain of callers.

Subpackages

Submodules

Package Contents

Classes

NameDescription
CallNodeOne node of the call tree.
CallRecorderRecord a context-keyed call tree from begin_call/end_call callbacks.
CallTreeCall tree of one recording.
SharedWeightA parameter or buffer a unit holds that code outside the unit still reads, looks up the device of, or moves.
WeightAccessOne access to a weight: which call or code did it, and how.

Functions

NameDescription
shared_weights_ofThe weights unit holds that something outside it still uses or moves.

API

class aitune.torch.CallNode(
label: str,
owner_class: str,
method: str,
module: torch.nn.Module | None = None,
calls: int = 0,
time_ms: float = 0.0,
bytes_in: int = 0,
bytes_out: int = 0,
children: list[aitune.torch.call_tree.nodes.NodeKey] = list(),
first_seen: int = 0
)
Dataclass

One node of the call tree.

bytes_in and bytes_out measure the tensor data passed into and returned from the node’s calls. Only tensors count, each as numel() * element_size(), and a tensor passed twice in one call counts once. Tensors inside lists, tuples, dicts and ModelOutput count too. Everything else counts as 0 bytes: numbers, strings, None, and objects such as a DynamicCache, including the tensors they hold.

Example

>>> node = CallNode( … key=(“root.forward”, “root.dit_forward”), label=“root.dit_forward”, owner_class=“Vla”, method=“dit_forward” … ) >>> node.path, node.parent_key (‘forward > dit_forward’, (‘root.forward’,))

bytes_in
int = 0

Tensor bytes in the arguments, summed over all recorded iterations. Bytes per call: bytes_in / calls.

bytes_out
int = 0

Tensor bytes in the results, summed over all recorded iterations.

calls
int = 0

Number of calls over all recorded iterations.

children
list[NodeKey] = field(default_factory=list)

Keys of the called nodes in first-call order.

first_seen
int = 0

Position of the node’s first call among all nodes.

key
NodeKey

Labels of the calling chain, outermost first. Identifies the node.

label
str

Adapter name of this call, for example root.vlm.model.visual.forward.

method
str

Name of the called method (forward for module calls).

module
Module | None = None

Module the method belongs to, or None for methods of plain objects.

owner_class
str

Class name of the object the method belongs to.

parent_key
NodeKey

Key of the calling node; empty for outermost calls.

path
str

The node key in written form (forward > dit_forward): its call labels, outermost first.

time_ms
float = 0.0

Time in milliseconds over all recorded iterations.

class aitune.torch.call_tree.CallRecorder()

Record a context-keyed call tree from begin_call/end_call callbacks.

Example: >>> recorder = CallRecorder() >>> with recorder: … recorder.begin_iteration() … outer = recorder.begin_call(“root.forward”, object(), “forward”, ()) … inner = recorder.begin_call(“root.head.forward”, object(), “forward”, ()) … recorder.end_call(inner) … recorder.end_call(outer) … recorder.end_iteration() >>> sorted(recorder.result().nodes) [(‘root.forward’,), (‘root.forward’, ‘root.head.forward’)]

_apply_depth
= 0
_current_sequence
list[NodeKey] | None = None
_events
dict[NodeKey, list[tuple[Any, Any]]] = {}
_modules
list[tuple[str, Module]] = []
_ms
dict[NodeKey, float] = {}
_nodes
dict[NodeKey, CallNode] = {}
_order
= 0
_original_apply
Callable[..., Any] | None = None
_paths
dict[NodeKey, str] = {}
_recording_now
CallRecorder | None = None
_registered
set[int] = set()
_sequences
list[list[NodeKey]] = []
_stack
list[NodeKey] = []
_use_cuda
= torch.cuda.is_available()
_weight_access_log
= WeightAccessLog()
_weight_access_mode
WeightAccessMode | None = None
active
bool

Whether the recorder is inside its with block.

aitune.torch.call_tree.CallRecorder.__enter__() -> typing.Self

Start recording: install the nn.Module._apply probe; calls are recorded until __exit__.

aitune.torch.call_tree.CallRecorder.__exit__(
exc_info: object = ()
) -> None

Stop recording and restore nn.Module._apply, also when the recorded code raised.

aitune.torch.call_tree.CallRecorder._make_apply_probe(
original_apply: collections.abc.Callable
) -> collections.abc.Callable

Build the replacement for nn.Module._apply that __enter__ installs.

nn.Module._apply is the method that .to(), .cuda(), .half() and similar calls use to convert a module’s parameters and buffers. The replacement runs original_apply unchanged. While the recorder is recording, it first records a "move" access on every tracked weight of the module and, unless recurse=False, of all its submodules: the innermost recorded call’s key, or the code found by unrecorded_caller if no recorded call is running. Only the outermost _apply of a conversion records, because _apply calls itself on each child. Recording goes through the storage map before original_apply runs, so a tied alias outside the moved subtree is moved as well, and the storage reads themselves do not count as accesses.

Parameters:

original_apply
Callable

The nn.Module._apply to call; restored by __exit__.

Returns: Callable

A function with the signature of nn.Module._apply.

aitune.torch.call_tree.CallRecorder._start_timer() -> tuple[typing.Any, int | None]

Start timing a call: record a CUDA event, or read time.perf_counter without CUDA.

Returns: tuple[Any, int | None]

The start of the timer and the CUDA device the start event was recorded on (None without CUDA).

aitune.torch.call_tree.CallRecorder._stop_timer(
) -> None

Stop timing a call: queue the CUDA event pair for flush, or add the CPU time to the node’s total.

aitune.torch.call_tree.CallRecorder.begin_call(
label: str,
obj: typing.Any,
method: str,
args: tuple[typing.Any, ...],
kwargs: dict[str, typing.Any] | None = None

Record the start of a call.

Parameters:

label
str

The adapter’s label of the call.

obj
Any

The object the method belongs to.

method
str

Method name.

args
tuple[Any, ...]

Positional arguments of the call.

kwargs
dict[str, Any] | NoneDefaults to None

Keyword arguments of the call; a callable among them makes the call transparent as well.

Returns: _Frame | None

A frame to pass to end_call, or None if the call is not recorded (not recording, exact torch container,

aitune.torch.call_tree.CallRecorder.begin_iteration() -> None

Mark the start of one iteration (for example one call of the inference function).

aitune.torch.call_tree.CallRecorder.end_call(
result: typing.Any = _NO_RESULT
) -> None

Record the end of a call started with begin_call.

Parameters:

frame
_Frame | None

The frame begin_call returned.

result
AnyDefaults to _NO_RESULT

The call’s return value, to count its tensor bytes; omit it when the call raised.

aitune.torch.call_tree.CallRecorder.end_iteration() -> None

Mark the end of the current iteration.

aitune.torch.call_tree.CallRecorder.flush() -> None

Resolve pending CUDA event pairs into the accumulated totals so they are not held until the end.

Adapters call it once per iteration; pending events stay bounded by one iteration. Each end event is synchronized on its own, so pairs recorded on any device resolve without a device-wide synchronization.

aitune.torch.call_tree.CallRecorder.register_module(
label_prefix: str,
module: torch.nn.Module
) -> None

Register an instrumented module for parameter names and sizes.

Parameters:

label_prefix
str

The adapter’s label for the module without the method, e.g. root.vlm.lm_head.

module
nn.Module

The module.

aitune.torch.call_tree.CallRecorder.result() -> aitune.torch.call_tree.call_tree.CallTree

Return the recorded call tree, with times resolved to milliseconds.

Idempotent: resolved event times accumulate into a per-key total, so calling result again returns the same times, and recording more calls afterwards keeps accumulating. The returned tree is a snapshot: its nodes are copies, so later recording does not change it.

class aitune.torch.CallTree(
iterations: int = 0,
sequences: list[list[aitune.torch.call_tree.nodes.NodeKey]] = list(),
weight_accesses: dict[str, set[aitune.torch.call_tree.weights.accesses.WeightAccess]] = dict(),
modules: list[tuple[str, torch.nn.Module]] = list()
)
Dataclass

Call tree of one recording.

Example

>>> outer = CallNode( … key=(“root.forward”,), label=“root.forward”, owner_class=“Vla”, method=“forward”, calls=2, time_ms=30.0 … ) >>> dit_module = nn.Linear(2, 2) >>> dit = CallNode( … key=(“root.forward”, “root.dit.forward”), … label=“root.dit.forward”, … owner_class=“Dit”, … method=“forward”, … module=dit_module, … calls=10, … time_ms=25.0, … first_seen=1, … ) >>> outer.children.append(dit.key) >>> tree = CallTree(nodes={n.key: n for n in (outer, dit)}, iterations=2) >>> [n.path for n in tree.roots], tree.total_ms ([‘forward’], 30.0) >>> [n.path for n in tree.descendants(outer)] [‘forward > dit.forward’] >>> [(n.path, n.calls) for n in tree.find(“dit”)] [(‘forward > dit.forward’, 10)] >>> [n.path for n in tree.nodes_of(dit_module)] [‘forward > dit.forward’]

iterations
int = 0

Number of recorded iterations.

modules
list[tuple[str, Module]] = field(default_factory=list)

Label prefix and module for every instrumented module, used for parameter names and sizes.

nodes
dict[NodeKey, CallNode] = field(default_factory=dict)

All nodes by key.

roots
list[CallNode]

Outermost nodes in first-call order.

sequences
list[list[NodeKey]] = field(default_factory=list)

Per iteration, the keys of the entered nodes in call order.

total_ms
float

Time of all outermost calls over all recorded iterations.

weight_accesses
dict[str, set[WeightAccess]] = field(default_factory=dict)

For each parameter or buffer (by weight name: its object path, e.g. vlm.lm_head.weight; a tied weight appears under each of its names), the WeightAccess records of accesses to it: reads, device checks and moves. Names stay valid when .to() or casts replace the tensors.

aitune.torch.CallTree.descendants(

All nodes below node.

aitune.torch.CallTree.find(
text: str

Nodes whose label contains text, in first-call order.

aitune.torch.CallTree.nodes_of(
module: torch.nn.Module,
method: str = 'forward'

Nodes recorded for calls of module.<method>.

A node is one chain of callers, so a module called from one place has one node, however often it ran there. A module called from several places (e.g. an LLM used by both prefill and generate) has one node per place.

Matching is by object identity (node.module is module), not by label, so a module that is reachable under several names (e.g. a tied embedding) is found under whichever name was recorded.

Parameters:

module
nn.Module

The module object that was recorded. After ait.wrap the attribute holds the wrapper instead, which matches nothing; pass a reference to the original module.

method
strDefaults to 'forward'

Name of the recorded method.

Returns: list[ CallNode ]

The matching nodes in first-call order; empty if the module was not recorded. Methods of objects that

aitune.torch.CallTree.to_dict() -> dict[str, typing.Any]

The call tree as plain JSON data, for tools that read it without torch or aitune.

Node values are totals over iterations: divide by iterations for per iteration, by calls for per call.

Example

>>> node = CallNode( … key=(“root.forward”, “root.dit.forward”), … label=“root.dit.forward”, … owner_class=“Dit”, … method=“forward”, … first_seen=1, … ) >>> outer = CallNode(key=(“root.forward”,), label=“root.forward”, owner_class=“Vla”, method=“forward”) >>> tree = CallTree(nodes={n.key: n for n in (outer, node)}, iterations=2) >>> exported = tree.to_dict()[“nodes”][1] >>> exported[“call_path”], exported[“owner_class”] (‘forward > dit.forward’, ‘Dit’) >>> exported[“call_path”].rpartition(” > ”)[2].rpartition(”.”) (‘dit’, ’.’, ‘forward’)

Returns: dict[str, Any]

A dict of str, int, float, list and dict values with these keys:

class aitune.torch.SharedWeight(
name: str,
aliases: list[str] = list(),
bytes: int = 0,
)
Dataclass

A parameter or buffer a unit holds that code outside the unit still reads, looks up the device of, or moves.

accesses
list[WeightAccess] = field(default_factory=list)

The accesses from outside the unit, sorted by (by, kind): reads, device lookups and moves. Unrecorded code is reported by its qualified name (SentenceTransformer.encode).

aliases
list[str] = field(default_factory=list)

Its other names, sorted (a tied lm_head.weight); empty if it has one name.

bytes
int = 0

numel * element_size. Views of one storage are each counted in full.

name
str

The weight’s name from the root, e.g. llm.embed.weight; a tied weight is named by its first name.

class aitune.torch.WeightAccess()

Bases: NamedTuple

One access to a weight: which call or code did it, and how.

by
str

The path of the recorded call it ran in (forward > llm.embed.forward, see CallNode.path); outside any recorded call, the code found by unrecorded_caller() (<unrecorded: Pipe.encode>) or <outside any call>. shared_weights_of reports unrecorded code by its qualified name (Pipe.encode).

kind
Literal['read', 'device', 'move']

"read" (an op uses its data), "device" (its device is checked: .device, is_cuda, get_device()), or "move" (nn.Module._apply: .to(), .cuda(), casts, reached it).

The weights unit holds that something outside it still uses or moves.

This is what tuning unit alone and offloading its weights would break: code outside it that reads one of its weights (a tied head, F.linear(x, unit.weight) in a caller’s forward), code that looks up a weight’s device, or code outside it that moves its modules with .to() or .cuda(). Reads and device lookups come from the ops recorded by WeightAccessMode during the call-tree pass; moves are recorded by the recorder’s nn.Module._apply probe.

Parameters:

tree
CallTree

CallTree — The call tree from ait.inspect(..., call_tree=True).

unit
UnitSpec

UnitSpec — A node path ("forward > dit_forward", see CallNode.path), a module (all its recorded forward calls), or a list of them as one unit.

Returns: list[ SharedWeight ]

One entry per weight, sorted by name; empty if the unit can be tuned and offloaded on its own.

Raises:

  • ValueError: If unit is empty, names a path with no recorded call, or is a module that is not part of the recorded model.
  • TypeError: If unit is neither a node path nor a module, nor a list of them.