Module adaptive_hints_intercept

Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-ffi.

AdaptiveHintsIntercept: opt-in LLM request intercept that injects AgentHints from HotCache trie.

This module provides AdaptiveHintsIntercept, which builds AgentHints from the prediction trie in HotCache and injects them into LLM request headers as a request intercept. AdaptiveHintsIntercept is opt-in and synchronously transforms the LlmRequest before it reaches the callable.

Structs

• AdaptiveHintsIntercept: Opt-in LLM request intercept that injects AgentHints into request headers from the prediction trie in HotCache.