Is this page helpful?

`core.inference.disaggregation.utils`#

Shared helpers for the disaggregation modules.

Module Contents#

Functions#

`intersect`	Overlap of two half-open `[lo, hi)` ranges, or `None` if disjoint.
`transfers_for_src`	Transfers in `plan` originating from `src_rank` (any KV/SSM reshard transfer – both expose a `src_rank` field).
`transfers_for_dst`	Transfers in `plan` destined for `dst_rank`.
`transfer_peer_records`	Normalize flat/TP/PP transfer metadata into peer/block records.
`transfer_block_count`	Return the sequence-block count represented by transfer metadata.

API#

core.inference.disaggregation.utils.intersect( a: Tuple[int, int], b: Tuple[int, int], ) → Optional[Tuple[int, int]]#: Overlap of two half-open [lo, hi) ranges, or None if disjoint.

core.inference.disaggregation.utils.transfers_for_src(plan, src_rank)#: Transfers in plan originating from src_rank (any KV/SSM reshard transfer – both expose a src_rank field).

core.inference.disaggregation.utils.transfers_for_dst(plan, dst_rank)#: Transfers in plan destined for dst_rank.

core.inference.disaggregation.utils.transfer_peer_records( peer_meta: Any, src_block_ids: List[int], ) → List[Tuple[dict, List[int]]]#: Normalize flat/TP/PP transfer metadata into peer/block records.

core.inference.disaggregation.utils.transfer_block_count( peer_meta: Any, src_block_ids: List[int], ) → int#: Return the sequence-block count represented by transfer metadata.