core.inference.routing

Stateless engine selection shared by ordinary and disaggregated routing.

Module Contents

Functions

select_engine

Select from an ordered pool without mutating its state.

API

core.inference.routing.select_engine(

identities: collections.abc.Sequence[bytes],

loads: numpy.ndarray,

*,

affinity_scores: numpy.ndarray | None = None,

available_fractions: numpy.ndarray | None = None,

routing_alpha: float = 0.0,

) → bytes

Select from an ordered pool without mutating its state.

All arrays align with identities; their order is the final tie-break. affinity_scores contains normalized reusable-work scores, or is None when there are no eligible affinity hits. Without hits, prefer available capacity (if supplied), then the lowest load. With hits, subtract the weighted relative load from affinity and break score ties by lowest load.