core.inference.data_parallel_inference_coordinator.handlers

Message handlers for the data parallel inference coordinator.

Each handler is a free function decorated with @message_handler, which records it in the module-level HANDLERS registry keyed by message header. The coordinator builds its dispatch table from this registry, so a new message type is supported simply by adding a decorated function here; the coordinator’s event loop never changes.

Handlers have the signature (coordinator, sender_identity, payload) -> bool | None where payload is the already-deserialized message. Returning a truthy value signals the coordinator’s event loop to stop.

Module Contents

Functions

message_handler	Register a function as the handler for one or more message headers.
handle_connect	Handshake with a new client, replying with a CONNECT_ACK.
handle_submit_request	Route a client request to a data parallel rank.
handle_control_signal	Validate a control signal against the transition table and broadcast it.
handle_cuda_profiler_signal	Broadcast a CUDA profiler control signal to every connected DP engine.
handle_engine_reply	Route completed requests from an engine back to their originating clients.
handle_engine_reply_partial	Route incremental engine replies without releasing request routing state.
handle_abort_request	Forward a client cancellation to the engine serving that request.
handle_shutdown	Stop the coordinator event loop on request from a known client.
handle_disconnect	Remove a disconnecting engine from the routing pool.

Data

HANDLERS

API

core.inference.data_parallel_inference_coordinator.handlers.HANDLERS

None

core.inference.data_parallel_inference_coordinator.handlers.message_handler(*headers)

Register a function as the handler for one or more message headers.

A new message type is supported by writing a handler function and decorating it with the header(s) it serves; it is added to HANDLERS, which the coordinator turns into its dispatch table. The event loop never needs to change when a header is added.

core.inference.data_parallel_inference_coordinator.handlers.handle_connect(coordinator, sender_identity, payload)

Handshake with a new client, replying with a CONNECT_ACK.

core.inference.data_parallel_inference_coordinator.handlers.handle_submit_request(coordinator, sender_identity, payload)

Route a client request to a data parallel rank.

Returns True (stopping the loop) if no engines are reachable.

core.inference.data_parallel_inference_coordinator.handlers.handle_control_signal(coordinator, sender_identity, payload)

Validate a control signal against the transition table and broadcast it.

core.inference.data_parallel_inference_coordinator.handlers.handle_cuda_profiler_signal(coordinator, sender_identity, payload)

Broadcast a CUDA profiler control signal to every connected DP engine.

Profiler control is not a coordinator state transition, so there are no CoordinatorState checks — the signal is simply forwarded to all engines.

core.inference.data_parallel_inference_coordinator.handlers.handle_engine_reply(coordinator, sender_identity, payload)

Route completed requests from an engine back to their originating clients.

core.inference.data_parallel_inference_coordinator.handlers.handle_engine_reply_partial(coordinator, sender_identity, payload)

Route incremental engine replies without releasing request routing state.

core.inference.data_parallel_inference_coordinator.handlers.handle_abort_request(coordinator, sender_identity, payload)

Forward a client cancellation to the engine serving that request.

core.inference.data_parallel_inference_coordinator.handlers.handle_shutdown(coordinator, sender_identity, payload)

Stop the coordinator event loop on request from a known client.

core.inference.data_parallel_inference_coordinator.handlers.handle_disconnect(coordinator, sender_identity, payload)

Remove a disconnecting engine from the routing pool.