> This page is for NeMo Relay, version dev.
> For other versions, use one of these documentation indexes:
> - Latest (v0.11.1) (default): https://docs.nvidia.com/nemo/relay/latest/llms.txt
> - dev: https://docs.nvidia.com/nemo/relay/dev/llms.txt
> - v0.11.1: https://docs.nvidia.com/nemo/relay/v0.11.1/llms.txt
> - v0.10.4: https://docs.nvidia.com/nemo/relay/v0.10.4/llms.txt
> - v0.9.4: https://docs.nvidia.com/nemo/relay/v0.9.4/llms.txt
> - v0.8.4: https://docs.nvidia.com/nemo/relay/v0.8.4/llms.txt
> - v0.7.3: https://docs.nvidia.com/nemo/relay/v0.7.3/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/relay/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/relay/_mcp/server.

# Module llm

> Managed and manual LLM calls.

Generated from `cargo doc --no-deps -p nemo-relay-builtins -p nemo-relay-diagnostics -p nemo-relay-logging -p nemo-relay-plugin-host -p nemo-relay-codecs -p nemo-relay -p nemo-relay-runtime -p nemo-relay-model-pricing -p nemo-relay-resource-metrics -p nemo-relay-observability -p nemo-relay-adaptive -p nemo-relay-pii-redaction -p nemo-relay-ffi -p nemo-relay-types -p nemo-relay-plugin -p nemo-relay-worker-proto -p nemo-relay-worker`.

Managed and manual LLM calls.

## Structs

* [LlmAttributes](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/struct-llmattributes): Bitflags that modify LLM-call behavior and observability.
* [LlmCallEndParams](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/struct-llmcallendparams): Builder parameters for [`llm_call_end`](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/fn-llm-call-end).
* [LlmCallExecuteParams](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/struct-llmcallexecuteparams): Builder parameters for [`llm_call_execute`](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/fn-llm-call-execute).
* [LlmCallParams](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/struct-llmcallparams): Builder parameters for [`llm_call`](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/fn-llm-call).
* [LlmHandle](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/struct-llmhandle): Runtime-owned handle identifying an active or completed LLM call.
* [LlmRequest](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/struct-llmrequest): JSON-shaped LLM request payload passed through the runtime.
* [LlmRequestInterceptOutcome](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/struct-llmrequestinterceptoutcome): Result of an LLM request intercept that can schedule lifecycle marks.
* [LlmStreamCallExecuteParams](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/struct-llmstreamcallexecuteparams): Builder parameters for [`llm_stream_call_execute`](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/fn-llm-stream-call-execute).

## Constants

* [LLM_REQUEST_INTERCEPT_OUTCOME_SCHEMA](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/constant-llm-request-intercept-outcome-schema): Versioned JSON envelope schema for [`LlmRequestInterceptOutcome`](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/struct-llmrequestinterceptoutcome).

## Functions

* [llm_call](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/fn-llm-call): Start a manual LLM lifecycle span.
* [llm_call_end](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/fn-llm-call-end): Finish a manual LLM lifecycle span.
* [llm_call_execute](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/fn-llm-call-execute): Execute an LLM call through the managed middleware pipeline.
* [llm_conditional_execution](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/fn-llm-conditional-execution): Run only the LLM conditional-execution guardrail chain.
* [llm_request_intercepts](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/fn-llm-request-intercepts): Run only the LLM request-intercept chain.
* [llm_stream_call_execute](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/llm/fn-llm-stream-call-execute): Execute a streaming LLM call through the managed middleware pipeline.