Struct OpenAI Chat Streaming Codec
Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-ffi.
Streaming counterpart to OpenAIChatCodec.
Replays the OpenAI Chat Completions SSE chunk sequence into the same JSON shape returned for a non-streaming request ({id, object, created, model, choices: [{message, finish_reason}], usage}). Once finalized, the assembled JSON can be fed back through OpenAIChatCodec::decode_response to produce the canonical AnnotatedLlmResponse.
Strategy
Chat Completions streams untyped SSE chunks of {choices: [{index, delta: {...}, finish_reason: ...}]}. Each delta may carry a role (typically only on the first chunk), incremental content text, or partial tool_calls whose function.arguments stream as a JSON-encoded string fragment-by-fragment. Top-level fields (id, model, created) are repeated on every chunk; we capture them once. Final-chunk usage is preserved when emitted (only sent when stream_options.include_usage is set on the request).
The OpenAI [DONE] end-of-stream sentinel is dropped by the SSE event decoder before reaching the collector, so this codec never sees it.
Internal state lives behind Arc<Mutex<...>> so the &self-produced collector and finalizer closures share access. Each instance is single-use because LlmFinalizerFn consumes the finalize step.
Implementations
impl OpenAIChatStreamingCodec
impl OpenAIChatStreamingCodecnew
pub fn new() -> SelfCreates a fresh streaming codec with empty accumulator state.
Trait Implementations
impl Default for OpenAIChatStreamingCodec
impl Default for OpenAIChatStreamingCodecdefault
fn default() -> Selfimpl StreamingCodec for OpenAIChatStreamingCodec
impl StreamingCodec for OpenAIChatStreamingCodeccollector
fn collector(&self) -> LlmCollectorFnfinalizer
fn finalizer(&self) -> LlmFinalizerFn