Is this page helpful?

core.inference.disaggregation.handoff_completion_tracker#

CPU control-plane aggregation for model-parallel handoff completion.

Module Contents#

Classes#

HandoffCompletionTracker

Aggregate per-rank transfer results at the model-parallel coordinator.

API#

class core.inference.disaggregation.handoff_completion_tracker.HandoffCompletionTracker(
zmq_context,
process_group: torch.distributed.ProcessGroup,
hostname: str | None = None,
)#

Aggregate per-rank transfer results at the model-parallel coordinator.

Initialization

_REPORT_FORMAT#

‘!qi?’

report(request_id: int, failed: bool) → None#

Report this rank’s terminal transfer result once for a request.

drain_completed() → list[tuple[int, bool]]#

Return requests that failed or completed on every model-parallel rank.

_record(request_id: int, rank: int, failed: bool) → None#