core.inference.disaggregation.decode_admission

Direct decode admission for imported prefill state.

Module Contents

Functions

additional_decode_blocks	Return blocks for verification and drafting beyond the imported prompt.
can_admit_prefilled_decode	Return whether an imported request can enter the live decode batch.
admit_prefilled_decode	Admit imported KV/SSM state directly as an active decode request.

API

core.inference.disaggregation.decode_admission.additional_decode_blocks(

prompt_length: int,

input_token_count: int,

block_size: int,

mtp_kv_cache: bool = False,

) → int

Return blocks for verification and drafting beyond the imported prompt.

core.inference.disaggregation.decode_admission.can_admit_prefilled_decode(context, input_token_count: int) → bool

Return whether an imported request can enter the live decode batch.

core.inference.disaggregation.decode_admission.admit_prefilled_decode(

context,

request,

prompt_block_ids: list[int],

continuation_block_ids: list[int],

input_tokens: list[int],

ssm_state_idx: int | None = None,

) → None

Admit imported KV/SSM state directly as an active decode request.

The imported prompt blocks already carry one allocator reference. This function transfers that ownership to the live request, binds the exact post-prompt recurrent state, and schedules the sampled first token (plus any MTP proposals) as the next decode query.