Struct Annotated LlmRequest | NVIDIA NeMo Relay

Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-ffi.

pub struct AnnotatedLlmRequest {Show 19 fields
    pub messages: Vec<Message>,
    pub model: Option<String>,
    pub params: Option<GenerationParams>,
    pub tools: Option<Vec<ToolDefinition>>,
    pub tool_choice: Option<ToolChoice>,
    pub store: Option<bool>,
    pub previous_response_id: Option<String>,
    pub truncation: Option<Json>,
    pub reasoning: Option<Json>,
    pub include: Option<Json>,
    pub user: Option<String>,
    pub metadata: Option<Json>,
    pub service_tier: Option<String>,
    pub parallel_tool_calls: Option<bool>,
    pub max_output_tokens: Option<u64>,
    pub max_tool_calls: Option<u64>,
    pub top_logprobs: Option<u64>,
    pub stream: Option<bool>,
    pub extra: Map<String, Json>,
}

Structured view of an LLM request, produced by a Codec from opaque LlmRequest content.

The extra field captures any provider-specific keys not modeled by the known fields, ensuring lossless round-trip through decode/encode.

Fields

`messages: Vec<Message>`

Parsed conversation messages.

`model: Option<String>`

Model identifier (e.g., "gpt-4", "claude-sonnet-4-20250514").

`params: Option<GenerationParams>`

Common generation parameters, normalized.

`tools: Option<Vec<ToolDefinition>>`

Tool definitions (function schemas) available to the model.

`tool_choice: Option<ToolChoice>`

Tool choice control.

`store: Option<bool>`

OpenAI Responses: whether to persist response state server-side.

`previous_response_id: Option<String>`

OpenAI Responses: prior response to continue from.

`truncation: Option<Json>`

OpenAI Responses: context truncation behavior.

`reasoning: Option<Json>`

OpenAI Responses: reasoning configuration object.

`include: Option<Json>`

OpenAI Responses: include filter for additional output/state items.

`user: Option<String>`

OpenAI user identifier.

`metadata: Option<Json>`

OpenAI metadata map/object.

`service_tier: Option<String>`

OpenAI service tier preference.

`parallel_tool_calls: Option<bool>`

OpenAI tool parallelism toggle.

`max_output_tokens: Option<u64>`

OpenAI Responses max output token limit.

`max_tool_calls: Option<u64>`

OpenAI Responses max tool calls.

`top_logprobs: Option<u64>`

OpenAI logprob fanout count.

`stream: Option<bool>`

OpenAI streaming toggle.

`extra: Map<String, Json>`

Extensible key-value pairs for unmodeled provider-specific fields. Merged back into the request body during encode via serde(flatten).

Implementations

`impl AnnotatedLlmRequest`

impl AnnotatedLlmRequest

`system_prompt`

pub fn system_prompt(&self) -> Option<&str>

Extract the text content of the first system message, if any.

For MessageContent::Text, returns the string directly. For MessageContent::Parts, returns the text of the first ContentPart::Text part.

`last_user_message`

pub fn last_user_message(&self) -> Option<&str>

Get the text content of the last user message, if any.

Searches messages in reverse order and returns the first user message found. For MessageContent::Parts, returns the text of the first ContentPart::Text part.

`has_tool_calls`

pub fn has_tool_calls(&self) -> bool

Check if any assistant message in the conversation contains tool calls.

Returns true if at least one Message::Assistant variant has a non-empty tool_calls field.

Trait Implementations

`impl Clone for AnnotatedLlmRequest`

impl Clone for AnnotatedLlmRequest

`clone`

fn clone(&self) -> AnnotatedLlmRequest

`clone_from`

fn clone_from(&mut self, source: &Self)

`impl Debug for AnnotatedLlmRequest`

impl Debug for AnnotatedLlmRequest

`fmt`

fn fmt(&self, f: &mut Formatter<'_>) -> Result

`impl<'de> Deserialize<'de> for AnnotatedLlmRequest`

impl<'de> Deserialize<'de> for AnnotatedLlmRequest

`deserialize`

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
    __D: Deserializer<'de>,

`impl PartialEq for AnnotatedLlmRequest`

impl PartialEq for AnnotatedLlmRequest

`eq`

fn eq(&self, other: &AnnotatedLlmRequest) -> bool

`ne`

fn ne(&self, other: &Rhs) -> bool

`impl Serialize for AnnotatedLlmRequest`

impl Serialize for AnnotatedLlmRequest

`serialize`

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>where
    __S: Serializer,

`impl StructuralPartialEq for AnnotatedLlmRequest`

impl StructuralPartialEq for AnnotatedLlmRequest