Struct Adaptive Hints Intercept

Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-pii-redaction -p nemo-relay-ffi -p nemo-relay-types -p nemo-relay-plugin -p nemo-relay-worker-proto -p nemo-relay-worker.

pub struct AdaptiveHintsIntercept { /* private fields */ }

Opt-in LLM request intercept that injects AgentHints into request headers from the prediction trie in HotCache.

Constructed via AdaptiveHintsIntercept::new and converted to an LlmRequestInterceptFn via AdaptiveHintsIntercept::into_request_fn for registration with the NeMo Relay runtime.

Implementations

impl AdaptiveHintsIntercept

impl AdaptiveHintsIntercept

new

pub fn new(hot_cache: Arc<RwLock<HotCache>>, agent_id: String) -> Self

Creates a new AdaptiveHintsIntercept that writes hints to the request header and to nvext.agent_hints in the body.

with_injection

pub fn with_injection(self, inject_header: bool, inject_body_path: &str) -> Self

Sets whether hints are added as a request header and the dot-separated body path they are written to.

into_request_fn

pub fn into_request_fn(self) -> LlmRequestInterceptFn

Converts this intercept into an LlmRequestInterceptFn suitable for registration with [register_llm_request_intercept].

The returned closure reads the HotCache trie, builds AgentHints, injects them into the request headers and body, and returns the transformed request.