Struct Adaptive Hints Intercept

Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-ffi.

1
pub struct AdaptiveHintsIntercept { /* private fields */ }

Opt-in LLM request intercept that injects AgentHints into request headers from the prediction trie in HotCache.

Constructed via AdaptiveHintsIntercept::new and converted to an LlmRequestInterceptFn via AdaptiveHintsIntercept::into_request_fn for registration with the NeMo Relay runtime.

Implementations

impl AdaptiveHintsIntercept

impl AdaptiveHintsIntercept

new

pub fn new(hot_cache: Arc<RwLock<HotCache>>, agent_id: String) -> Self

Creates a new AdaptiveHintsIntercept.

into_request_fn

pub fn into_request_fn(self) -> LlmRequestInterceptFn

Converts this intercept into an LlmRequestInterceptFn suitable for registration with [register_llm_request_intercept].

The returned closure reads the HotCache trie, builds AgentHints, injects them into the request headers and body, and returns the transformed request.