> This page is for NeMo Relay, version dev.
> For other versions, use one of these documentation indexes:
> - Latest (v0.11.1) (default): https://docs.nvidia.com/nemo/relay/latest/llms.txt
> - dev: https://docs.nvidia.com/nemo/relay/dev/llms.txt
> - v0.11.1: https://docs.nvidia.com/nemo/relay/v0.11.1/llms.txt
> - v0.10.4: https://docs.nvidia.com/nemo/relay/v0.10.4/llms.txt
> - v0.9.4: https://docs.nvidia.com/nemo/relay/v0.9.4/llms.txt
> - v0.8.4: https://docs.nvidia.com/nemo/relay/v0.8.4/llms.txt
> - v0.7.3: https://docs.nvidia.com/nemo/relay/v0.7.3/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/relay/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/relay/_mcp/server.

# Struct Annotated LlmRequest

> Normalized request structure with explicit provider-specific and extension fields.

Generated from `cargo doc --no-deps -p nemo-relay-builtins -p nemo-relay-diagnostics -p nemo-relay-logging -p nemo-relay-plugin-host -p nemo-relay-codecs -p nemo-relay -p nemo-relay-runtime -p nemo-relay-model-pricing -p nemo-relay-resource-metrics -p nemo-relay-observability -p nemo-relay-adaptive -p nemo-relay-pii-redaction -p nemo-relay-ffi -p nemo-relay-types -p nemo-relay-plugin -p nemo-relay-worker-proto -p nemo-relay-worker`.

<pre />

Normalized request structure with explicit provider-specific and extension fields.

## Fields

### `messages: Vec<LlmMessage>`

Parsed conversation messages.

### `instructions: Option<MessageContent>`

Provider-level instructions that are not part of the conversation array. Anthropic encodes this as `system`; OpenAI Responses uses `instructions`.

### `model: Option<String>`

Model identifier (e.g., `"gpt-4"`, `"claude-sonnet-4-20250514"`).

### `params: Option<GenerationParams>`

Common generation parameters, normalized.

### `tools: Option<ToolDefinitionList>`

Tool definitions (function schemas) available to the model.

### `tool_choice: Option<ToolChoice>`

Tool choice control.

### `store: Option<bool>`

OpenAI Responses: whether to persist response state server-side.

### `previous_response_id: Option<String>`

OpenAI Responses: prior response to continue from.

### `truncation: Option<JsonValue>`

OpenAI Responses: context truncation behavior.

### `reasoning: Option<JsonValue>`

OpenAI Responses: reasoning configuration object.

### `include: Option<JsonValue>`

OpenAI Responses: include filter for additional output/state items.

### `user: Option<String>`

OpenAI user identifier.

### `metadata: Option<JsonValue>`

OpenAI metadata map/object.

### `service_tier: Option<String>`

OpenAI service tier preference.

### `parallel_tool_calls: Option<bool>`

OpenAI tool parallelism toggle.

### `max_output_tokens: Option<u64>`

OpenAI Responses max output token limit.

### `max_tool_calls: Option<u64>`

OpenAI Responses max tool calls.

### `top_logprobs: Option<u64>`

OpenAI logprob fanout count.

### `stream: Option<bool>`

OpenAI streaming toggle.

### `api_specific: Option<JsonValue>`

Codec-owned request data; preserved as JSON and validated by the selected codec.

### `extra: HashMap<String, JsonValue>`

Unknown provider fields, excluding modeled field names.

## Implementations

### `impl AnnotatedLlmRequest`

<pre />

#### `model`

<pre />

Returns the value of `model`, or the default value if `model` is unset.

#### `store`

<pre />

Returns the value of `store`, or the default value if `store` is unset.

#### `previous_response_id`

<pre />

Returns the value of `previous_response_id`, or the default value if `previous_response_id` is unset.

#### `user`

<pre />

Returns the value of `user`, or the default value if `user` is unset.

#### `service_tier`

<pre />

Returns the value of `service_tier`, or the default value if `service_tier` is unset.

#### `parallel_tool_calls`

<pre />

Returns the value of `parallel_tool_calls`, or the default value if `parallel_tool_calls` is unset.

#### `max_output_tokens`

<pre />

Returns the value of `max_output_tokens`, or the default value if `max_output_tokens` is unset.

#### `max_tool_calls`

<pre />

Returns the value of `max_tool_calls`, or the default value if `max_tool_calls` is unset.

#### `top_logprobs`

<pre />

Returns the value of `top_logprobs`, or the default value if `top_logprobs` is unset.

#### `stream`

<pre />

Returns the value of `stream`, or the default value if `stream` is unset.

## Trait Implementations

### `impl Clone for AnnotatedLlmRequest`

<pre />

#### `clone`

<pre />

#### `clone_from`

<pre />

### `impl Debug for AnnotatedLlmRequest`

<pre />

#### `fmt`

<pre />

### `impl Default for AnnotatedLlmRequest`

<pre />

#### `default`

<pre />

### `impl Message for AnnotatedLlmRequest`

<pre />

#### `encoded_len`

<pre />

#### `clear`

<pre />

#### `encode`

<pre />

#### `encode_to_vec`

<pre />

#### `encode_length_delimited`

<pre />

#### `encode_length_delimited_to_vec`

<pre />

#### `decode`

<pre />

#### `decode_length_delimited`

<pre />

#### `merge`

<pre />

#### `merge_length_delimited`

<pre />

### `impl Name for AnnotatedLlmRequest`

<pre />

#### `NAME`

<pre />

#### `PACKAGE`

<pre />

#### `full_name`

<pre />

#### `type_url`

<pre />

### `impl PartialEq for AnnotatedLlmRequest`

<pre />

#### `eq`

<pre />

#### `ne`

<pre />

### `impl WireValue<AnnotatedLlmRequest> for AnnotatedLlmRequest`

<pre />

#### `to_wire`

<pre />

#### `from_wire`

<pre />

### `impl StructuralPartialEq for AnnotatedLlmRequest`

<pre />