ReferenceAPIsRust Library Referencenemo-relay-fficallable

Struct Nemo Relay LlmExecution Context

View as Markdown

Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-pii-redaction -p nemo-relay-ffi -p nemo-relay-types -p nemo-relay-plugin -p nemo-relay-worker-proto -p nemo-relay-worker.

#[repr(C)]pub struct NemoRelayLlmExecutionContext {
    pub request_codec: NemoRelayLlmRequestContext,
    pub response_codec: *const NemoRelayLlmResponseContext,
}

Directional codec context supplied to an LLM execution intercept.

request_codec is always present. response_codec is non-null for unary execution and null for streaming execution, where Relay has no completed response to decode. Pointers reachable from this value are borrowed and valid only until the intercept callback returns.

Fields

request_codec: NemoRelayLlmRequestContext

Active request codec identity and capability.

response_codec: *const NemoRelayLlmResponseContext

Active unary-response codec context, or null for streaming execution.