core.inference.disaggregation.handoff_completion_tracker#
CPU control-plane aggregation for model-parallel handoff completion.
Module Contents#
Classes#
Aggregate per-rank transfer results at the model-parallel coordinator. |
API#
- class core.inference.disaggregation.handoff_completion_tracker.HandoffCompletionTracker(
- zmq_context,
- process_group: torch.distributed.ProcessGroup,
- hostname: str | None = None,
Aggregate per-rank transfer results at the model-parallel coordinator.
Initialization
- _REPORT_FORMAT#
‘!qi??’
- report(request_id: int, failed: bool, source_safe: bool) None#
Report this rank’s transfer result and source-storage safety.
- drain_completed() list[tuple[int, bool, bool]]#
Return requests that failed or completed on every model-parallel rank.
- _record(
- request_id: int,
- rank: int,
- failed: bool,
- source_safe: bool,