Skip to navigation
ReferenceAPIsRust Library Referencenemo-relaymiddlewarecallbacks

Enum LlmCollector

View as Markdown

Generated from cargo doc --no-deps -p nemo-relay-builtins -p nemo-relay-diagnostics -p nemo-relay-logging -p nemo-relay-plugin-host -p nemo-relay-codecs -p nemo-relay -p nemo-relay-runtime -p nemo-relay-model-pricing -p nemo-relay-resource-metrics -p nemo-relay-observability -p nemo-relay-adaptive -p nemo-relay-pii-redaction -p nemo-relay-ffi -p nemo-relay-types -p nemo-relay-plugin -p nemo-relay-worker-proto -p nemo-relay-worker.

pub enum LlmCollector {
    Sync(Box<dyn FnMut(Value) -> Result<(), FlowError> + Send>),
    Async(Box<dyn FnMut(Value) -> Pin<Box<dyn Future<Output = Result<(), FlowError>> + Send>> + Send>),
}

Collector used to accumulate streaming response chunks.

An asynchronous collector still pending at close() is awaited after the producer closes, so it must settle once the producer does. Any FnMut(Json) -> Result<()> converts into the synchronous form.

Variants

Sync(Box<dyn FnMut(Value) -> Result<(), FlowError> + Send>)

Sync(Box<dyn FnMut(Value) -> Result<(), FlowError> + Send>)

Runs inline while the stream is polled.

Async(Box<dyn FnMut(Value) -> Pin<Box<dyn Future<Output = Result<(), FlowError>> + Send>> + Send>)

Async(Box<dyn FnMut(Value) -> Pin<Box<dyn Future<Output = Result<(), FlowError>> + Send>> + Send>)

Awaited before its chunk is yielded.

Implementations

impl LlmCollector

impl LlmCollector

new_async

pub fn new_async<F, Fut>(collector: F) -> LlmCollectorwhere
    F: FnMut(Value) -> Fut + Send + 'static,
    Fut: Future<Output = Result<(), FlowError>> + Send + 'static,

Wrap a future-returning closure as an asynchronous collector.

Trait Implementations

impl<F> From<F> for LlmCollectorwhere F: FnMut(Value) -> Result<(), FlowError> + Send + 'static,

impl<F> From<F> for LlmCollectorwhere
    F: FnMut(Value) -> Result<(), FlowError> + Send + 'static,

from

fn from(collector: F) -> LlmCollector