Trait LlmCodec

Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-ffi.

pub trait LlmCodec: Send + Sync {
    // Required methods
    fn decode(&self, request: &LlmRequest) -> Result<AnnotatedLlmRequest>;
    fn encode(
        &self,
        annotated: &AnnotatedLlmRequest,
        original: &LlmRequest,
    ) -> Result<LlmRequest>;
}

A bidirectional translator between opaque LlmRequest content and structured AnnotatedLlmRequest.

Codecs are implemented by integration patches (LangChain, LangChain-NVIDIA, LangGraph, etc.) since each SDK has its own request format. They are registered by name in the global codec registry.

Design

• Synchronous: decode/encode are pure data transforms (JSON restructuring), not I/O operations. This matches existing guardrails and request intercepts.
• Send + Sync: Required because NemoRelayContextState is behind Arc<RwLock<>> and accessed from async contexts.
• Trait object: Codecs are registered at runtime (e.g., by Python patches), so the Rust core cannot know concrete types at compile time. Store as Arc<dyn LlmCodec>.

Required Methods

decode

fn decode(&self, request: &LlmRequest) -> Result<AnnotatedLlmRequest>

Parse opaque request content into structured form.

encode

fn encode(
    &self,
    annotated: &AnnotatedLlmRequest,
    original: &LlmRequest,
) -> Result<LlmRequest>

Merge structured changes back into the opaque request.

The original parameter is the pre-intercept LlmRequest, used to preserve fields that the Codec does not structurally model. Implementations MUST use merge-not-replace semantics: overlay structured changes onto the original content, do not construct a fresh content object.

Implementors

impl LlmCodec for AnthropicMessagesCodec

impl LlmCodec for AnthropicMessagesCodec

impl LlmCodec for OpenAIChatCodec

impl LlmCodec for OpenAIChatCodec

impl LlmCodec for OpenAIResponsesCodec

impl LlmCodec for OpenAIResponsesCodec