Trait Streaming Codec

Generated from cargo doc --no-deps -p nemo-relay-builtins -p nemo-relay-diagnostics -p nemo-relay-logging -p nemo-relay-plugin-host -p nemo-relay-codecs -p nemo-relay -p nemo-relay-runtime -p nemo-relay-model-pricing -p nemo-relay-resource-metrics -p nemo-relay-observability -p nemo-relay-adaptive -p nemo-relay-pii-redaction -p nemo-relay-ffi -p nemo-relay-types -p nemo-relay-plugin -p nemo-relay-worker-proto -p nemo-relay-worker.

pub trait StreamingCodec: Send + Sync {
    // Required methods
    fn collector(&self) -> Box<dyn FnMut(Value) -> Result<(), FlowError> + Send>;
    fn finalizer(&self) -> Box<dyn FnOnce() -> Value + Send>;
​
    // Provided methods
    fn validate_stream(&self, stream: LlmJsonStream) -> LlmJsonStream { ... }
    fn borrowed_collector(
        &self,
    ) -> Option<Box<dyn FnMut(&Value) -> Result<(), FlowError> + Send>> { ... }
}

Per-provider streaming codec used with the runtime streaming execution API.

collector() and finalizer() produce owned closures that share the codec’s internal accumulation state. Implementations typically wrap that state in Arc<Mutex<...>> so each &self-produced closure captures a clone of the handle.

LlmFinalizerFn is FnOnce, so a StreamingCodec instance is single-use: callers construct a fresh instance per managed-lifecycle call and discard it after the stream completes.

Required Methods

collector

fn collector(&self) -> Box<dyn FnMut(Value) -> Result<(), FlowError> + Send>

Returns a closure that consumes one decoded provider event per call.

finalizer

fn finalizer(&self) -> Box<dyn FnOnce() -> Value + Send>

Returns a closure that, when called once at end of stream, produces the assembled response payload in the shape the matching crate::codec::traits::LlmResponseCodec can decode.

Provided Methods

validate_stream

fn validate_stream(&self, stream: LlmJsonStream) -> LlmJsonStream

Wraps the source stream to check provider completion rules.

Feed the returned stream to this instance’s collector and finalizer. The default leaves the stream unchanged. An override must forward close and cleanup operations.

borrowed_collector

fn borrowed_collector(
    &self,
) -> Option<Box<dyn FnMut(&Value) -> Result<(), FlowError> + Send>>

Returns an optional collector that reads decoded events by reference.

Pass this to LlmStreamCallExecuteParams::borrowed_collector to avoid copying each whole JSON chunk. The default keeps the owned callback for custom codecs.

Implementations on Foreign Types

impl StreamingCodec for AnthropicMessagesStreamingCodec

impl StreamingCodec for AnthropicMessagesStreamingCodec

collector

fn collector(&self) -> Box<dyn FnMut(Value) -> Result<(), FlowError> + Send>

borrowed_collector

fn borrowed_collector(
    &self,
) -> Option<Box<dyn FnMut(&Value) -> Result<(), FlowError> + Send>>

finalizer

fn finalizer(&self) -> Box<dyn FnOnce() -> Value + Send>

impl StreamingCodec for GeminiGenerateContentStreamingCodec

impl StreamingCodec for GeminiGenerateContentStreamingCodec

collector

fn collector(&self) -> Box<dyn FnMut(Value) -> Result<(), FlowError> + Send>

borrowed_collector

fn borrowed_collector(
    &self,
) -> Option<Box<dyn FnMut(&Value) -> Result<(), FlowError> + Send>>

finalizer

fn finalizer(&self) -> Box<dyn FnOnce() -> Value + Send>

impl StreamingCodec for OCIGenAIStreamingCodec

impl StreamingCodec for OCIGenAIStreamingCodec

collector

fn collector(&self) -> Box<dyn FnMut(Value) -> Result<(), FlowError> + Send>

borrowed_collector

fn borrowed_collector(
    &self,
) -> Option<Box<dyn FnMut(&Value) -> Result<(), FlowError> + Send>>

finalizer

fn finalizer(&self) -> Box<dyn FnOnce() -> Value + Send>

impl StreamingCodec for OpenAIChatStreamingCodec

impl StreamingCodec for OpenAIChatStreamingCodec

collector

fn collector(&self) -> Box<dyn FnMut(Value) -> Result<(), FlowError> + Send>

borrowed_collector

fn borrowed_collector(
    &self,
) -> Option<Box<dyn FnMut(&Value) -> Result<(), FlowError> + Send>>

finalizer

fn finalizer(&self) -> Box<dyn FnOnce() -> Value + Send>

impl StreamingCodec for OpenAIResponsesStreamingCodec

impl StreamingCodec for OpenAIResponsesStreamingCodec

validate_stream

fn validate_stream(&self, stream: LlmJsonStream) -> LlmJsonStream

Reports an error at EOF if this codec’s collector did not receive a final response.

collector

fn collector(&self) -> Box<dyn FnMut(Value) -> Result<(), FlowError> + Send>

Collects response snapshots and output items into this codec’s shared state.

finalizer

fn finalizer(&self) -> Box<dyn FnOnce() -> Value + Send>

Takes the collected state and returns the final provider response.

borrowed_collector

fn borrowed_collector(
    &self,
) -> Option<Box<dyn FnMut(&Value) -> Result<(), FlowError> + Send>>

Implementors