core.inference.disaggregation.handoff_ownership#
Source-owned handoffs: bounded offers and explicitly acknowledged readers.
Module Contents#
Classes#
Wire result of a claim; only expired, unclaimed offers may be retried. |
|
Keep ownership beside the source allocation, independently of decode workers. |
API#
- class core.inference.disaggregation.handoff_ownership.HandoffClaimStatus#
Bases:
str,enum.EnumWire result of a claim; only expired, unclaimed offers may be retried.
Initialization
Initialize self. See help(type(self)) for accurate signature.
- ACCEPTED#
‘accepted’
- EXPIRED#
‘expired’
- REJECTED#
‘rejected’
- class core.inference.disaggregation.handoff_ownership._Handoff#
- engine: bytes#
None
- expires_at: float | None#
None
- owner: str | None#
None
- class core.inference.disaggregation.handoff_ownership.HandoffOwnership(offer_timeout: float = 300.0)#
Keep ownership beside the source allocation, independently of decode workers.
A supervisor may retire an owner only after its adapter and all decode ranks have exited and cannot restart. Losing discovery or a heartbeat is not proof.
Initialization
- offer(
- request_id: int,
- engine: bytes,
- *,
- require_claim: bool = False,
Register source state; only claim-aware consumers permit offer expiry.
- claim(
- request_id: int,
- owner: str,
Claim once before reading; duplicate consumers must not share a release.
- expired_unclaimed() list[tuple[int, bytes]]#
List expired offers for release, retaining failed deliveries for retry.
Claims are fenced by the same deadline, even before the coordinator’s next sweep. Claimed reads NEVER expire: time is not a completion barrier.
- source_engine(request_id: int) bytes | None#
Return the source without forgetting a release that may need retrying.
- release(request_id: int) bytes | None#
Forget a completed handoff and return its source engine.
- confirm_terminated(owner: str) list[tuple[int, bytes]]#
Fence delayed claims and list state to release after successful delivery.