Struct HotCache | NVIDIA NeMo Relay

Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-ffi.

pub struct HotCache {
    pub plan: Option<ExecutionPlan>,
    pub trie: Option<PredictionTrieNode>,
    pub agent_hints_default: Option<AgentHints>,
    pub acg_profiles: HashMap<String, StabilityAnalysisResult>,
    pub acg_profile_observation_counts: HashMap<String, u32>,
    pub acg_stability: Option<StabilityAnalysisResult>,
    pub acg_observation_count: u32,
}

In-memory cache of adaptive artifacts needed on the hot path.

The adaptive runtime keeps this structure in an std::sync::RwLock so intercepts and event-processing tasks can exchange recently learned plans, trie state, and Adaptive Cache Governor (ACG) summaries without hitting the configured backend on every request.

Fields

`plan: Option<ExecutionPlan>`

Current execution plan used for tool parallelism hints.

`trie: Option<PredictionTrieNode>`

Prediction trie used to derive default latency sensitivity hints.

`agent_hints_default: Option<AgentHints>`

Default agent-level hints computed from the prediction trie.

`acg_profiles: HashMap<String, StabilityAnalysisResult>`

Per-profile ACG stability results keyed by derived profile identifier.

`acg_profile_observation_counts: HashMap<String, u32>`

Observation counts corresponding to entries in Self::acg_profiles.

`acg_stability: Option<StabilityAnalysisResult>`

Aggregate ACG stability result used for warm-first eligibility checks.

`acg_observation_count: u32`

Observation count associated with Self::acg_stability.

Trait Implementations

`impl Clone for HotCache`

impl Clone for HotCache

`clone`

fn clone(&self) -> HotCache

`clone_from`

fn clone_from(&mut self, source: &Self)

`impl Debug for HotCache`

impl Debug for HotCache

`fmt`

fn fmt(&self, f: &mut Formatter<'_>) -> Result

`impl<'de> Deserialize<'de> for HotCache`

impl<'de> Deserialize<'de> for HotCache

`deserialize`

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
    __D: Deserializer<'de>,

`impl Serialize for HotCache`

impl Serialize for HotCache

`serialize`

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>where
    __S: Serializer,