Struct OCIGenAI Streaming Codec

Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-pii-redaction -p nemo-relay-ffi -p nemo-relay-types -p nemo-relay-plugin -p nemo-relay-worker-proto -p nemo-relay-worker.

1 pub struct OCIGenAIStreamingCodec { /* private fields */ }

Streaming counterpart to OCIGenAIChatCodec.

Replays the OCI Generative AI SSE event sequence into the same JSON shape a non-streaming ChatResult carries ({modelId, chatResponse: {apiFormat, ...}}). Once finalized, the assembled JSON can be fed back through OCIGenAIChatCodec::decode_response to produce an AnnotatedLlmResponse - meaning streaming and non-streaming OCI requests converge on the same observability output.

Strategy

OCI streams untagged chat-response deltas. GENERIC events carry {index, message: {role, content: [{type: "TEXT", text}], toolCalls}, finishReason} fragments whose text and tool-call arguments accumulate per choice index; COHERE events carry incremental {apiFormat: "COHERE", text} fragments with finishReason on the terminal event. Events wrapped in a chatResponse envelope are unwrapped first, and modelId/usage are captured whenever a chunk supplies them.

Internal state lives behind Arc<Mutex<...>> so the &self-produced collector and finalizer closures share access. Each instance is single-use because LlmFinalizerFn consumes the finalize step.

Implementations

`impl OCIGenAIStreamingCodec`

impl OCIGenAIStreamingCodec

`new`

pub fn new() -> Self

Creates a fresh streaming codec with empty accumulator state.

Trait Implementations

`impl Default for OCIGenAIStreamingCodec`

impl Default for OCIGenAIStreamingCodec

`default`

fn default() -> Self

`impl StreamingCodec for OCIGenAIStreamingCodec`

impl StreamingCodec for OCIGenAIStreamingCodec

`collector`

fn collector(&self) -> LlmCollectorFn

`finalizer`

fn finalizer(&self) -> LlmFinalizerFn