Type Alias LlmExecution Fn
Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-ffi.
pub type LlmExecutionFn = Arc<dyn Fn(&str, LlmRequest, LlmExecutionNextFn) -> Pin<Box<dyn Future<Output = Result<Json>> + Send>> + Send + Sync>;Wrap or replace non-streaming LLM execution.
A non-streaming execution intercept receives the logical provider name, the current request, and the continuation representing the rest of the chain.
Parameters
- First argument: Logical provider or model family name.
- Second argument: Current LLM request.
- Third argument: Continuation for the remaining execution chain.
Returns
A future resolving to the provider response JSON.
Errors
The future resolves to an error when the intercept or remaining execution chain fails.