Trait LlmResponse Codec

Generated from cargo doc --no-deps -p nemo-relay-builtins -p nemo-relay-diagnostics -p nemo-relay-logging -p nemo-relay-plugin-host -p nemo-relay-codecs -p nemo-relay -p nemo-relay-runtime -p nemo-relay-model-pricing -p nemo-relay-resource-metrics -p nemo-relay-observability -p nemo-relay-adaptive -p nemo-relay-pii-redaction -p nemo-relay-ffi -p nemo-relay-types -p nemo-relay-plugin -p nemo-relay-worker-proto -p nemo-relay-worker.

pub trait LlmResponseCodec: Send + Sync {
    // Required method
    fn decode_response(
        &self,
        response: &Value,
    ) -> Result<AnnotatedLlmResponse, FlowError>;
​
    // Provided methods
    fn codec_identity(&self) -> LlmCodecIdentity { ... }
    fn allows_estimated_cost(&self, _response: &Value) -> bool { ... }
}

Decode-only codec for LLM API responses.

Unlike LlmCodec (which is bidirectional for requests), response codecs are introspection-only: they parse a raw response into structured form but never need to encode back. This matches the pipeline design where responses are observed, not modified.

Design

• Synchronous: decode_response is a pure data transform (JSON parsing), not an I/O operation.
• Send + Sync: Required for storage in Arc behind RwLock.
• Trait object: Codecs are registered at runtime, stored as Arc<dyn LlmResponseCodec>.
• Fallible: Returns Result; managed call sites may omit annotations on decode failure, while manual lifecycle bindings may surface the error.

Two-Phase Decode

Implementations should use a two-phase decode pattern:

1. Deserialize raw JSON into API-specific intermediate structs
2. Map intermediate structs into the normalized AnnotatedLlmResponse

Required Methods

decode_response

fn decode_response(
    &self,
    response: &Value,
) -> Result<AnnotatedLlmResponse, FlowError>

Parse a raw JSON response into normalized structured form.

Implementations should return Err only for genuinely unparseable input.

Provided Methods

codec_identity

fn codec_identity(&self) -> LlmCodecIdentity

Return this codec’s identity for LLM sanitizer context.

Custom codecs should keep the default LlmCodecIdentity::Opaque unless they have a stable runtime registration ID.

allows_estimated_cost

fn allows_estimated_cost(&self, _response: &Value) -> bool

Return whether aggregate response usage can be priced from its response model.

Most providers report usage for one model, so the default permits model-based cost enrichment. A provider can opt out when its response combines usage from multiple models and no provider-reported cost is available.

Implementations on Foreign Types

impl LlmResponseCodec for AnthropicMessagesCodec

impl LlmResponseCodec for AnthropicMessagesCodec

codec_identity

fn codec_identity(&self) -> LlmCodecIdentity

allows_estimated_cost

fn allows_estimated_cost(&self, response: &Value) -> bool

decode_response

fn decode_response(
    &self,
    response: &Value,
) -> Result<AnnotatedLlmResponse, FlowError>

impl LlmResponseCodec for GeminiGenerateContentCodec

impl LlmResponseCodec for GeminiGenerateContentCodec

codec_identity

fn codec_identity(&self) -> LlmCodecIdentity

decode_response

fn decode_response(
    &self,
    response: &Value,
) -> Result<AnnotatedLlmResponse, FlowError>

impl LlmResponseCodec for OCIGenAIChatCodec

impl LlmResponseCodec for OCIGenAIChatCodec

codec_identity

fn codec_identity(&self) -> LlmCodecIdentity

decode_response

fn decode_response(
    &self,
    response: &Value,
) -> Result<AnnotatedLlmResponse, FlowError>

impl LlmResponseCodec for OpenAIChatCodec

impl LlmResponseCodec for OpenAIChatCodec

codec_identity

fn codec_identity(&self) -> LlmCodecIdentity

decode_response

fn decode_response(
    &self,
    response: &Value,
) -> Result<AnnotatedLlmResponse, FlowError>

impl LlmResponseCodec for OpenAIResponsesCodec

impl LlmResponseCodec for OpenAIResponsesCodec

codec_identity

fn codec_identity(&self) -> LlmCodecIdentity

decode_response

fn decode_response(
    &self,
    response: &Value,
) -> Result<AnnotatedLlmResponse, FlowError>

Implementors