aitune.torch.call_tree
aitune.torch.call_tree
Call tree: which recorded calls ran inside which, keyed by their chain of callers.
Subpackages
Submodules
aitune.torch.call_tree.call_treeaitune.torch.call_tree.callersaitune.torch.call_tree.methodsaitune.torch.call_tree.nodesaitune.torch.call_tree.recorderaitune.torch.call_tree.syncaitune.torch.call_tree.units
Package Contents
Classes
Functions
API
One node of the call tree.
bytes_in and bytes_out measure the tensor data passed into and returned from the node’s calls. Only
tensors count, each as numel() * element_size(), and a tensor passed twice in one call counts once. Tensors
inside lists, tuples, dicts and ModelOutput count too. Everything else counts as 0 bytes: numbers, strings,
None, and objects such as a DynamicCache, including the tensors they hold.
Example
>>> node = CallNode( … key=(“root.forward”, “root.dit_forward”), label=“root.dit_forward”, owner_class=“Vla”, method=“dit_forward” … ) >>> node.path, node.parent_key (‘forward > dit_forward’, (‘root.forward’,))
Tensor bytes in the arguments, summed over all recorded iterations. Bytes per call:
bytes_in / calls.
Tensor bytes in the results, summed over all recorded iterations.
Number of calls over all recorded iterations.
Keys of the called nodes in first-call order.
Position of the node’s first call among all nodes.
Labels of the calling chain, outermost first. Identifies the node.
Adapter name of this call, for example root.vlm.model.visual.forward.
Name of the called method (forward for module calls).
Module the method belongs to, or None for methods of plain objects.
Class name of the object the method belongs to.
Key of the calling node; empty for outermost calls.
The node key in written form (forward > dit_forward): its call labels, outermost first.
Time in milliseconds over all recorded iterations.
Record a context-keyed call tree from begin_call/end_call callbacks.
Example: >>> recorder = CallRecorder() >>> with recorder: … recorder.begin_iteration() … outer = recorder.begin_call(“root.forward”, object(), “forward”, ()) … inner = recorder.begin_call(“root.head.forward”, object(), “forward”, ()) … recorder.end_call(inner) … recorder.end_call(outer) … recorder.end_iteration() >>> sorted(recorder.result().nodes) [(‘root.forward’,), (‘root.forward’, ‘root.head.forward’)]
Whether the recorder is inside its with block.
Start recording: install the nn.Module._apply probe; calls are recorded until __exit__.
Stop recording and restore nn.Module._apply, also when the recorded code raised.
Build the replacement for nn.Module._apply that __enter__ installs.
nn.Module._apply is the method that .to(), .cuda(), .half() and similar calls use to convert a
module’s parameters and buffers. The replacement runs original_apply unchanged. While the recorder is
recording, it first records a "move" access on every tracked weight of the module and, unless
recurse=False, of all its submodules: the innermost recorded call’s key, or the code found by
unrecorded_caller if no recorded call is running. Only the outermost _apply of a conversion records,
because _apply calls itself on each child. Recording goes through the storage map before
original_apply runs, so a tied alias outside the moved subtree is moved as well, and the storage reads
themselves do not count as accesses.
Parameters:
The nn.Module._apply to call; restored by __exit__.
Returns: Callable
A function with the signature of nn.Module._apply.
Start timing a call: record a CUDA event, or read time.perf_counter without CUDA.
Returns: tuple[Any, int | None]
The start of the timer and the CUDA device the start event was recorded on (None without CUDA).
Stop timing a call: queue the CUDA event pair for flush, or add the CPU time to the node’s total.
Record the start of a call.
Parameters:
The adapter’s label of the call.
The object the method belongs to.
Method name.
Positional arguments of the call.
Keyword arguments of the call; a callable among them makes the call transparent as well.
Returns: _Frame | None
A frame to pass to end_call, or None if the call is not recorded (not recording, exact torch container,
Mark the start of one iteration (for example one call of the inference function).
Record the end of a call started with begin_call.
Parameters:
The frame begin_call returned.
The call’s return value, to count its tensor bytes; omit it when the call raised.
Mark the end of the current iteration.
Resolve pending CUDA event pairs into the accumulated totals so they are not held until the end.
Adapters call it once per iteration; pending events stay bounded by one iteration. Each end event is synchronized on its own, so pairs recorded on any device resolve without a device-wide synchronization.
Register an instrumented module for parameter names and sizes.
Parameters:
The adapter’s label for the module without the method, e.g. root.vlm.lm_head.
The module.
Return the recorded call tree, with times resolved to milliseconds.
Idempotent: resolved event times accumulate into a per-key total, so calling result again returns the
same times, and recording more calls afterwards keeps accumulating. The returned tree is a snapshot: its nodes
are copies, so later recording does not change it.
Call tree of one recording.
Example
>>> outer = CallNode( … key=(“root.forward”,), label=“root.forward”, owner_class=“Vla”, method=“forward”, calls=2, time_ms=30.0 … ) >>> dit_module = nn.Linear(2, 2) >>> dit = CallNode( … key=(“root.forward”, “root.dit.forward”), … label=“root.dit.forward”, … owner_class=“Dit”, … method=“forward”, … module=dit_module, … calls=10, … time_ms=25.0, … first_seen=1, … ) >>> outer.children.append(dit.key) >>> tree = CallTree(nodes={n.key: n for n in (outer, dit)}, iterations=2) >>> [n.path for n in tree.roots], tree.total_ms ([‘forward’], 30.0) >>> [n.path for n in tree.descendants(outer)] [‘forward > dit.forward’] >>> [(n.path, n.calls) for n in tree.find(“dit”)] [(‘forward > dit.forward’, 10)] >>> [n.path for n in tree.nodes_of(dit_module)] [‘forward > dit.forward’]
Number of recorded iterations.
Label prefix and module for every instrumented module, used for parameter names and sizes.
All nodes by key.
Outermost nodes in first-call order.
Per iteration, the keys of the entered nodes in call order.
Time of all outermost calls over all recorded iterations.
For each parameter or buffer (by weight name: its object path, e.g.
vlm.lm_head.weight; a tied weight appears under each of its names), the WeightAccess records of
accesses to it: reads, device checks and moves. Names stay valid when .to() or casts replace the
tensors.
All nodes below node.
Nodes whose label contains text, in first-call order.
Nodes recorded for calls of module.<method>.
A node is one chain of callers, so a module called from one place has one node, however often it ran there.
A module called from several places (e.g. an LLM used by both prefill and generate) has one node per
place.
Matching is by object identity (node.module is module), not by label, so a module that is reachable under
several names (e.g. a tied embedding) is found under whichever name was recorded.
Parameters:
The module object that was recorded. After ait.wrap the attribute holds the wrapper instead,
which matches nothing; pass a reference to the original module.
Name of the recorded method.
Returns: list[ CallNode ]
The matching nodes in first-call order; empty if the module was not recorded. Methods of objects that
The call tree as plain JSON data, for tools that read it without torch or aitune.
Node values are totals over iterations: divide by iterations for per iteration, by calls for
per call.
Example
>>> node = CallNode( … key=(“root.forward”, “root.dit.forward”), … label=“root.dit.forward”, … owner_class=“Dit”, … method=“forward”, … first_seen=1, … ) >>> outer = CallNode(key=(“root.forward”,), label=“root.forward”, owner_class=“Vla”, method=“forward”) >>> tree = CallTree(nodes={n.key: n for n in (outer, node)}, iterations=2) >>> exported = tree.to_dict()[“nodes”][1] >>> exported[“call_path”], exported[“owner_class”] (‘forward > dit.forward’, ‘Dit’) >>> exported[“call_path”].rpartition(” > ”)[2].rpartition(”.”) (‘dit’, ’.’, ‘forward’)
Returns: dict[str, Any]
A dict of str, int, float, list and dict values with these keys:
A parameter or buffer a unit holds that code outside the unit still reads, looks up the device of, or moves.
The accesses from outside the unit, sorted by (by, kind): reads, device lookups and moves.
Unrecorded code is reported by its qualified name (SentenceTransformer.encode).
Its other names, sorted (a tied lm_head.weight); empty if it has one name.
numel * element_size. Views of one storage are each counted in full.
The weight’s name from the root, e.g. llm.embed.weight; a tied weight is named by its first name.
Bases: NamedTuple
One access to a weight: which call or code did it, and how.
The path of the recorded call it ran in (forward > llm.embed.forward, see CallNode.path); outside
any recorded call, the code found by unrecorded_caller() (<unrecorded: Pipe.encode>) or
<outside any call>. shared_weights_of reports unrecorded code by its qualified name
(Pipe.encode).
"read" (an op uses its data), "device" (its device is checked: .device, is_cuda,
get_device()), or "move" (nn.Module._apply: .to(), .cuda(), casts, reached it).
The weights unit holds that something outside it still uses or moves.
This is what tuning unit alone and offloading its weights would break: code outside it that reads one of its
weights (a tied head, F.linear(x, unit.weight) in a caller’s forward), code that looks up a weight’s device,
or code outside it that moves its modules with .to() or .cuda(). Reads and device lookups come from the
ops recorded by WeightAccessMode during the call-tree pass; moves are recorded by the recorder’s
nn.Module._apply probe.
Parameters:
CallTree — The call tree from ait.inspect(..., call_tree=True).
UnitSpec — A node path ("forward > dit_forward", see CallNode.path), a module (all its recorded
forward calls), or a list of them as one unit.
Returns: list[ SharedWeight ]
One entry per weight, sorted by name; empty if the unit can be tuned and offloaded on its own.
Raises:
ValueError: Ifunitis empty, names a path with no recorded call, or is a module that is not part of the recorded model.TypeError: Ifunitis neither a node path nor a module, nor a list of them.