Skip to navigation
ReferenceAPIsRust Library Referencenemo-relayllm

Function llm_call

View as Markdown

Generated from cargo doc --no-deps -p nemo-relay-builtins -p nemo-relay-diagnostics -p nemo-relay-logging -p nemo-relay-plugin-host -p nemo-relay-codecs -p nemo-relay -p nemo-relay-runtime -p nemo-relay-model-pricing -p nemo-relay-resource-metrics -p nemo-relay-observability -p nemo-relay-adaptive -p nemo-relay-pii-redaction -p nemo-relay-ffi -p nemo-relay-types -p nemo-relay-plugin -p nemo-relay-worker-proto -p nemo-relay-worker.

pub fn llm_call(params: LlmCallParams<'_>) -> Result<LlmHandle, FlowError>

Start a manual LLM lifecycle span.

This emits an LLM-start event after applying sanitize-request guardrails to the payload recorded for observability.

If a sanitizer errors or panics, Relay omits the payload and request annotation and does not run remaining sanitizers.

Parameters

  • name: Logical provider or model family name recorded on the span.
  • request: Raw LlmRequest associated with the span.
  • parent: Optional explicit parent scope.
  • attributes: LLM attribute bitflags applied to the span.
  • data: Optional application payload stored on the returned handle. The emitted start event data is the sanitized request payload.
  • metadata: Optional JSON metadata recorded on the start event.
  • model_name: Optional normalized model name recorded separately from the request payload.
  • annotated_request: Optional normalized request annotation produced by a codec.
  • codec: Optional codec for sanitizing the request copy saved in telemetry. Relay also uses it to create or update that copy’s request annotation.
  • timestamp: Optional timestamp recorded as the handle start time and on the emitted start event. When None, the current UTC time is used.

Returns

A Result containing the created LlmHandle after its start-event snapshot has been submitted for queued publication.

Errors

Returns an error when the runtime owner check fails or when internal state cannot be read safely. Dispatcher submission failures are logged because observability publication is best effort.

Notes

The runtime removes standard credential headers (authorization, proxy-authorization, cookie, x-api-key, api-key, anthropic-api-key, and x-goog-api-key) from the event-only request copy before sanitize-request guardrails run. This does not change the caller-owned LlmRequest. By default, Relay records the full request history for the first LLM call in an agent scope and the first call after compaction. For later calls, Relay limits the request annotation to the current user turn. With a request codec, Relay applies the same limit to the request copy saved in the event. Both managed and manual calls use this rule. The provider request stays unchanged. Set the observability plugin’s enable_full_payloads option to keep the full request history in events.