Trait LlmResponse Codec | NVIDIA NeMo Relay

Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-ffi.

pub trait LlmResponseCodec: Send + Sync {
    // Required method
    fn decode_response(&self, response: &Json) -> Result<AnnotatedLlmResponse>;
}

Decode-only codec for LLM API responses.

Unlike LlmCodec (which is bidirectional for requests), response codecs are introspection-only: they parse a raw response into structured form but never need to encode back. This matches the pipeline design where responses are observed, not modified.

Design

Synchronous: decode_response is a pure data transform (JSON parsing), not an I/O operation.
Send + Sync: Required for storage in Arc behind RwLock.
Trait object: Codecs are registered at runtime, stored as Arc<dyn LlmResponseCodec>.
Fallible: Returns Result; managed call sites may omit annotations on decode failure, while manual lifecycle bindings may surface the error.

Two-Phase Decode

Implementations should use a two-phase decode pattern:

Deserialize raw JSON into API-specific intermediate structs
Map intermediate structs into the normalized AnnotatedLlmResponse

Required Methods

`decode_response`

fn decode_response(&self, response: &Json) -> Result<AnnotatedLlmResponse>

Parse a raw JSON response into normalized structured form.

Implementations should return Err only for genuinely unparseable input.

Implementors

`impl LlmResponseCodec for AnthropicMessagesCodec`

impl LlmResponseCodec for AnthropicMessagesCodec

`impl LlmResponseCodec for OpenAIChatCodec`

impl LlmResponseCodec for OpenAIChatCodec

`impl LlmResponseCodec for OpenAIResponsesCodec`

impl LlmResponseCodec for OpenAIResponsesCodec