Skip to navigation
ReferenceAPIsRust Library Referencenemo-relaycodectraits

Trait LlmCodec

View as Markdown

Generated from cargo doc --no-deps -p nemo-relay-builtins -p nemo-relay-diagnostics -p nemo-relay-logging -p nemo-relay-plugin-host -p nemo-relay-codecs -p nemo-relay -p nemo-relay-runtime -p nemo-relay-model-pricing -p nemo-relay-resource-metrics -p nemo-relay-observability -p nemo-relay-adaptive -p nemo-relay-pii-redaction -p nemo-relay-ffi -p nemo-relay-types -p nemo-relay-plugin -p nemo-relay-worker-proto -p nemo-relay-worker.

pub trait LlmCodec: Send + Sync {
    // Required methods
    fn decode(
        &self,
        request: &LlmRequest,
    ) -> Result<AnnotatedLlmRequest, FlowError>;
    fn encode(
        &self,
        annotated: &AnnotatedLlmRequest,
        original: &LlmRequest,
    ) -> Result<LlmRequest, FlowError>;
​
    // Provided methods
    fn codec_identity(&self) -> LlmCodecIdentity { ... }
    fn encode_with_baseline(
        &self,
        annotated: &AnnotatedLlmRequest,
        original: &LlmRequest,
        _baseline: &AnnotatedLlmRequest,
    ) -> Result<LlmRequest, FlowError> { ... }
    fn encode_owned_with_baseline(
        &self,
        annotated: &AnnotatedLlmRequest,
        original: LlmRequest,
        baseline: &AnnotatedLlmRequest,
    ) -> Result<LlmRequest, FlowError> { ... }
}

A bidirectional translator between opaque LlmRequest content and structured AnnotatedLlmRequest.

Codecs are implemented by framework integrations and provider adapters because each SDK has its own request format. A codec is supplied per call by the caller; the built-in provider codecs can also be selected from a raw payload by the runtime provider resolver.

Design

  • Synchronous: decode/encode are pure data transforms (JSON restructuring), not I/O operations. This matches existing guardrails and request intercepts.
  • Send + Sync: Required for codec handles shared across async runtime contexts.
  • Trait object: Codecs are registered at runtime by callers or bindings, so the Rust core cannot know concrete types at compile time. Store as Arc<dyn LlmCodec>.

Required Methods

decode

fn decode(&self, request: &LlmRequest) -> Result<AnnotatedLlmRequest, FlowError>

Parse opaque request content into structured form.

encode

fn encode(
    &self,
    annotated: &AnnotatedLlmRequest,
    original: &LlmRequest,
) -> Result<LlmRequest, FlowError>

Merge structured changes back into the opaque request.

The original parameter is the pre-intercept LlmRequest, used to preserve fields that the Codec does not structurally model. Implementations MUST use merge-not-replace semantics: overlay structured changes onto the original content, do not construct a fresh content object.

Provided Methods

codec_identity

fn codec_identity(&self) -> LlmCodecIdentity

Return this codec’s identity for LLM sanitizer context.

Custom codecs should keep the default LlmCodecIdentity::Opaque unless they have a stable runtime registration ID. Callers must not infer a provider surface from an opaque implementation or request shape.

encode_with_baseline

fn encode_with_baseline(
    &self,
    annotated: &AnnotatedLlmRequest,
    original: &LlmRequest,
    _baseline: &AnnotatedLlmRequest,
) -> Result<LlmRequest, FlowError>

Merge changes using the annotation decoded from original.content.

Use a baseline from this codec and the same raw request body. Do not change the baseline or that body after decoding. Request headers can change while middleware runs. The runtime supplies this baseline to avoid decoding twice. The default calls Self::encode, so existing custom codecs still work.

encode_owned_with_baseline

fn encode_owned_with_baseline(
    &self,
    annotated: &AnnotatedLlmRequest,
    original: LlmRequest,
    baseline: &AnnotatedLlmRequest,
) -> Result<LlmRequest, FlowError>

Encode a request that the caller gives to this codec.

The codec can move the request body or keep it for later use. The baseline must match the original request body. If both header maps have the same key, keep the value from the original request. The default calls Self::encode_with_baseline and then moves the original headers into the result.

Implementations on Foreign Types

impl LlmCodec for AnthropicMessagesCodec

impl LlmCodec for AnthropicMessagesCodec

codec_identity

fn codec_identity(&self) -> LlmCodecIdentity

decode

fn decode(&self, request: &LlmRequest) -> Result<AnnotatedLlmRequest, FlowError>

encode

fn encode(
    &self,
    annotated: &AnnotatedLlmRequest,
    original: &LlmRequest,
) -> Result<LlmRequest, FlowError>

encode_with_baseline

fn encode_with_baseline(
    &self,
    annotated: &AnnotatedLlmRequest,
    original: &LlmRequest,
    baseline: &AnnotatedLlmRequest,
) -> Result<LlmRequest, FlowError>

encode_owned_with_baseline

fn encode_owned_with_baseline(
    &self,
    annotated: &AnnotatedLlmRequest,
    original: LlmRequest,
    baseline: &AnnotatedLlmRequest,
) -> Result<LlmRequest, FlowError>

impl LlmCodec for GeminiGenerateContentCodec

impl LlmCodec for GeminiGenerateContentCodec

codec_identity

fn codec_identity(&self) -> LlmCodecIdentity

decode

fn decode(&self, request: &LlmRequest) -> Result<AnnotatedLlmRequest, FlowError>

encode

fn encode(
    &self,
    annotated: &AnnotatedLlmRequest,
    original: &LlmRequest,
) -> Result<LlmRequest, FlowError>

encode_with_baseline

fn encode_with_baseline(
    &self,
    annotated: &AnnotatedLlmRequest,
    original: &LlmRequest,
    baseline: &AnnotatedLlmRequest,
) -> Result<LlmRequest, FlowError>

encode_owned_with_baseline

fn encode_owned_with_baseline(
    &self,
    annotated: &AnnotatedLlmRequest,
    original: LlmRequest,
    baseline: &AnnotatedLlmRequest,
) -> Result<LlmRequest, FlowError>

impl LlmCodec for OCIGenAIChatCodec

impl LlmCodec for OCIGenAIChatCodec

codec_identity

fn codec_identity(&self) -> LlmCodecIdentity

decode

fn decode(&self, request: &LlmRequest) -> Result<AnnotatedLlmRequest, FlowError>

encode

fn encode(
    &self,
    annotated: &AnnotatedLlmRequest,
    original: &LlmRequest,
) -> Result<LlmRequest, FlowError>

encode_with_baseline

fn encode_with_baseline(
    &self,
    annotated: &AnnotatedLlmRequest,
    original: &LlmRequest,
    baseline: &AnnotatedLlmRequest,
) -> Result<LlmRequest, FlowError>

encode_owned_with_baseline

fn encode_owned_with_baseline(
    &self,
    annotated: &AnnotatedLlmRequest,
    original: LlmRequest,
    baseline: &AnnotatedLlmRequest,
) -> Result<LlmRequest, FlowError>

impl LlmCodec for OpenAIChatCodec

impl LlmCodec for OpenAIChatCodec

codec_identity

fn codec_identity(&self) -> LlmCodecIdentity

decode

fn decode(&self, request: &LlmRequest) -> Result<AnnotatedLlmRequest, FlowError>

encode

fn encode(
    &self,
    annotated: &AnnotatedLlmRequest,
    original: &LlmRequest,
) -> Result<LlmRequest, FlowError>

encode_with_baseline

fn encode_with_baseline(
    &self,
    annotated: &AnnotatedLlmRequest,
    original: &LlmRequest,
    baseline: &AnnotatedLlmRequest,
) -> Result<LlmRequest, FlowError>

encode_owned_with_baseline

fn encode_owned_with_baseline(
    &self,
    annotated: &AnnotatedLlmRequest,
    original: LlmRequest,
    baseline: &AnnotatedLlmRequest,
) -> Result<LlmRequest, FlowError>

impl LlmCodec for OpenAIResponsesCodec

impl LlmCodec for OpenAIResponsesCodec

codec_identity

fn codec_identity(&self) -> LlmCodecIdentity

decode

fn decode(&self, request: &LlmRequest) -> Result<AnnotatedLlmRequest, FlowError>

encode

fn encode(
    &self,
    annotated: &AnnotatedLlmRequest,
    original: &LlmRequest,
) -> Result<LlmRequest, FlowError>

encode_with_baseline

fn encode_with_baseline(
    &self,
    annotated: &AnnotatedLlmRequest,
    original: &LlmRequest,
    baseline: &AnnotatedLlmRequest,
) -> Result<LlmRequest, FlowError>

encode_owned_with_baseline

fn encode_owned_with_baseline(
    &self,
    annotated: &AnnotatedLlmRequest,
    original: LlmRequest,
    baseline: &AnnotatedLlmRequest,
) -> Result<LlmRequest, FlowError>

Implementors