Type Alias LlmStream Execution Fn
Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-ffi.
pub type LlmStreamExecutionFn = Arc<dyn Fn(&str, LlmRequest, LlmStreamExecutionNextFn) -> Pin<Box<dyn Future<Output = Result<LlmJsonStream>> + Send>> + Send + Sync>;Wrap or replace streaming LLM execution.
A streaming execution intercept can observe or modify the request before invoking the continuation, and it can also replace the returned stream.
Parameters
- First argument: Logical provider or model family name.
- Second argument: Current LLM request.
- Third argument: Continuation for the remaining streaming execution chain.
Returns
A future resolving to a JSON chunk stream.
Errors
The future resolves to an error when the intercept or remaining streaming execution chain fails.