Struct Agent Hints | NVIDIA NeMo Relay

Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-ffi.

pub struct AgentHints {
    pub osl: u32,
    pub iat: u32,
    pub priority: i32,
    pub latency_sensitivity: f64,
    pub prefix_id: String,
    pub total_requests: u32,
}

Runtime hint bundle exposed to downstream integrations.

These values summarize the current learned default behavior for an agent and are suitable for transport in provider-specific headers or metadata fields.

Fields

`osl: u32`

Output-size limit hint in tokens.

`iat: u32`

Inter-arrival-time hint in milliseconds.

`priority: i32`

Scheduling priority hint derived from latency sensitivity.

`latency_sensitivity: f64`

Learned latency sensitivity score for the current prefix.

`prefix_id: String`

Identifier of the prefix or trie node the hints came from.

`total_requests: u32`

Estimated total number of requests in the workflow.

Trait Implementations

`impl Clone for AgentHints`

impl Clone for AgentHints

`clone`

fn clone(&self) -> AgentHints

`clone_from`

fn clone_from(&mut self, source: &Self)

`impl Debug for AgentHints`

impl Debug for AgentHints

`fmt`

fn fmt(&self, f: &mut Formatter<'_>) -> Result

`impl<'de> Deserialize<'de> for AgentHints`

impl<'de> Deserialize<'de> for AgentHints

`deserialize`

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
    __D: Deserializer<'de>,

`impl Serialize for AgentHints`

impl Serialize for AgentHints

`serialize`

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>where
    __S: Serializer,