Type Alias LlmStream Execution Next Fn

Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-ffi.

pub type LlmStreamExecutionNextFn = Arc<dyn Fn(LlmRequest) -> Pin<Box<dyn Future<Output = Result<LlmJsonStream>> + Send>> + Send + Sync>;

Continuation type invoked by streaming LLM execution intercepts.

This callable represents the remainder of the streaming LLM execution chain and resolves to a stream of JSON response chunks.

Parameters

• First argument: LLM request to pass to the remaining streaming execution chain.

Returns

A future resolving to a JSON chunk stream.

Errors

The future resolves to an error when the remaining streaming execution chain fails.

Aliased Type

1
pub struct LlmStreamExecutionNextFn { /* private fields */ }