> This page is for NeMo Relay, version v0.10.0.
> For other versions, use one of these documentation indexes:
> - Latest (v0.10.0) (default): https://docs.nvidia.com/nemo/relay/latest/llms.txt
> - dev: https://docs.nvidia.com/nemo/relay/dev/llms.txt
> - v0.10.0: https://docs.nvidia.com/nemo/relay/v0.10.0/llms.txt
> - v0.9.4: https://docs.nvidia.com/nemo/relay/v0.9.4/llms.txt
> - v0.8.4: https://docs.nvidia.com/nemo/relay/v0.8.4/llms.txt
> - v0.7.3: https://docs.nvidia.com/nemo/relay/v0.7.3/llms.txt
> - v0.6.0: https://docs.nvidia.com/nemo/relay/v0.6.0/llms.txt
> - v0.5.0: https://docs.nvidia.com/nemo/relay/v0.5.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/relay/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/relay/_mcp/server.

# nemo-relay-worker

> Rust SDK for NeMo Relay out-of-process gRPC worker plugins.

Generated from `cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-pii-redaction -p nemo-relay-ffi -p nemo-relay-types -p nemo-relay-plugin -p nemo-relay-worker-proto -p nemo-relay-worker`.

Rust SDK for NeMo Relay out-of-process gRPC worker plugins.

## Invocation cancellation

The `grpc-v1` service tracks active unary and streaming callbacks by the host-provided invocation ID. Relay sends `CancelInvocation` when a managed caller is cancelled, an invocation times out, or a host stream is abandoned. The SDK aborts the matching async callback task and reports `worker.cancelled`; cancellation of an unknown, completed, or already cancelled ID returns a negative acknowledgment.

Cancellation is cooperative. Dropping an async callback future releases its Rust-owned resources, but an accepted acknowledgment does not prove that arbitrary blocking work started by the callback has stopped.

## Macros

* [relay\_debug](/reference/api/rust-library-reference/nemo-relay-worker/macro-relay-debug): Emit a formatted debug-level gRPC worker record; await the returned future.
* [relay\_error](/reference/api/rust-library-reference/nemo-relay-worker/macro-relay-error): Emit a formatted error-level gRPC worker record; await the returned future.
* [relay\_info](/reference/api/rust-library-reference/nemo-relay-worker/macro-relay-info): Emit a formatted info-level gRPC worker record; await the returned future.
* [relay\_trace](/reference/api/rust-library-reference/nemo-relay-worker/macro-relay-trace): Emit a formatted trace-level gRPC worker record; await the returned future.
* [relay\_warn](/reference/api/rust-library-reference/nemo-relay-worker/macro-relay-warn): Emit a formatted warn-level gRPC worker record; await the returned future.

## Structs

* [AnnotatedLlmRequest](/reference/api/rust-library-reference/nemo-relay-worker/struct-annotatedllmrequest): Structured view of an LLM request, produced by a Codec from opaque [`LlmRequest`](/reference/api/rust-library-reference/nemo-relay-worker/struct-llmrequest) content.
* [AnnotatedLlmResponse](/reference/api/rust-library-reference/nemo-relay-worker/struct-annotatedllmresponse): Structured view of an LLM response, produced by a response codec from raw JSON API output.
* [ConditionalMiddlewareGuardrailHandle](/reference/api/rust-library-reference/nemo-relay-worker/struct-conditionalmiddlewareguardrailhandle): Opaque activation-owned key for removing a worker-created gate.
* [ConfigDiagnostic](/reference/api/rust-library-reference/nemo-relay-worker/struct-configdiagnostic): Structured diagnostic reported while validating or activating plugin configuration.
* [DataSchema](/reference/api/rust-library-reference/nemo-relay-worker/struct-dataschema): Identifier for the schema that describes an event's opaque `data` payload.
* [EmitMarkOptions](/reference/api/rust-library-reference/nemo-relay-worker/struct-emitmarkoptions): Optional fields supported by the additive `grpc-v1` mark-emission request.
* [EventCategory](/reference/api/rust-library-reference/nemo-relay-worker/struct-eventcategory): Semantic category carried by ATOF `category`.
* [EventSanitizeFields](/reference/api/rust-library-reference/nemo-relay-worker/struct-eventsanitizefields): Observability fields that event sanitizers may rewrite.
* [LlmExecutionContext](/reference/api/rust-library-reference/nemo-relay-worker/struct-llmexecutioncontext): Invocation-scoped codec identities and operations for an LLM execution interceptor.
* [LlmNext](/reference/api/rust-library-reference/nemo-relay-worker/struct-llmnext): Continuation handle for LLM execution intercepts.
* [LlmOptimizationContribution](/reference/api/rust-library-reference/nemo-relay-worker/struct-llmoptimizationcontribution): One optimizer's evidence for a change to an LLM call.
* [LlmOptimizationKind](/reference/api/rust-library-reference/nemo-relay-worker/struct-llmoptimizationkind): Open, forward-compatible optimization classification.
* [LlmOptimizationModel](/reference/api/rust-library-reference/nemo-relay-worker/struct-llmoptimizationmodel): Model identity used for counterfactual and effective pricing.
* [LlmOptimizationModelTransition](/reference/api/rust-library-reference/nemo-relay-worker/struct-llmoptimizationmodeltransition): A model change proposed or applied by an optimizer.
* [LlmOptimizationSummary](/reference/api/rust-library-reference/nemo-relay-worker/struct-llmoptimizationsummary): Close-time accounting attached to the normalized LLM response.
* [LlmOptimizationTokenImpact](/reference/api/rust-library-reference/nemo-relay-worker/struct-llmoptimizationtokenimpact): Shared token evidence that Relay can aggregate without understanding a plugin payload.
* [LlmOptimizationTokens](/reference/api/rust-library-reference/nemo-relay-worker/struct-llmoptimizationtokens): Token quantities retained independently from pricing arithmetic.
* [LlmRequest](/reference/api/rust-library-reference/nemo-relay-worker/struct-llmrequest): JSON-shaped LLM request payload passed through the runtime.
* [LlmRequestInterceptOutcome](/reference/api/rust-library-reference/nemo-relay-worker/struct-llmrequestinterceptoutcome): Result of an LLM request intercept that can schedule lifecycle marks.
* [LlmSanitizeRequestContext](/reference/api/rust-library-reference/nemo-relay-worker/struct-llmsanitizerequestcontext): Active codec identity and capability for an LLM request.
* [LlmSanitizeResponseContext](/reference/api/rust-library-reference/nemo-relay-worker/struct-llmsanitizeresponsecontext): Active codec identity and capability for an LLM response.
* [LlmStreamNext](/reference/api/rust-library-reference/nemo-relay-worker/struct-llmstreamnext): Continuation handle for LLM stream execution intercepts.
* [MetricEnvelope](/reference/api/rust-library-reference/nemo-relay-worker/struct-metricenvelope): Payload stored in a mark using [`METRIC_DATA_SCHEMA_NAME`](/reference/api/rust-library-reference/nemo-relay-worker/constant-metric-data-schema-name).
* [MetricMeasurement](/reference/api/rust-library-reference/nemo-relay-worker/struct-metricmeasurement): One recording operation in a Relay metric mark.
* [PendingMarkSpec](/reference/api/rust-library-reference/nemo-relay-worker/struct-pendingmarkspec): Mark requested by middleware for materialization by a lifecycle owner.
* [PluginContext](/reference/api/rust-library-reference/nemo-relay-worker/struct-plugincontext): Registration context passed to [`WorkerPlugin::register`](/reference/api/rust-library-reference/nemo-relay-worker/trait-workerplugin).
* [PluginRuntime](/reference/api/rust-library-reference/nemo-relay-worker/struct-pluginruntime): Cloneable handle for calling the Relay host runtime from worker callbacks.
* [RuntimeDiagnostic](/reference/api/rust-library-reference/nemo-relay-worker/struct-runtimediagnostic): One bounded runtime diagnostic reported by the Relay host.
* [RuntimeDiagnostics](/reference/api/rust-library-reference/nemo-relay-worker/struct-runtimediagnostics): Bounded snapshot of active host runtime diagnostics.
* [RuntimeRegistrationIdentity](/reference/api/rust-library-reference/nemo-relay-worker/struct-runtimeregistrationidentity): Structured identity for a global gateable runtime registration.
* [RuntimeRegistrationOwner](/reference/api/rust-library-reference/nemo-relay-worker/struct-runtimeregistrationowner): Discovery metadata describing the owner of a runtime registration.
* [ToolExecutionContext](/reference/api/rust-library-reference/nemo-relay-worker/struct-toolexecutioncontext): Per-call context supplied to a tool execution intercept.
* [ToolExecutionInterceptOutcome](/reference/api/rust-library-reference/nemo-relay-worker/struct-toolexecutioninterceptoutcome): Canonical result returned by a tool execution intercept.
* [ToolExecutionResult](/reference/api/rust-library-reference/nemo-relay-worker/struct-toolexecutionresult): Canonical application-visible result of tool execution.
* [ToolNext](/reference/api/rust-library-reference/nemo-relay-worker/struct-toolnext): Continuation handle for tool execution intercepts.
* [WorkerRequestCodec](/reference/api/rust-library-reference/nemo-relay-worker/struct-workerrequestcodec): Invocation-scoped proxy for the active LLM request codec.
* [WorkerResponseCodec](/reference/api/rust-library-reference/nemo-relay-worker/struct-workerresponsecodec): Invocation-scoped proxy for the active LLM response codec.
* [WorkerServerConfig](/reference/api/rust-library-reference/nemo-relay-worker/struct-workerserverconfig): Explicit worker server configuration for tests and custom launchers.

## Enums

* [BuiltinLlmCodec](/reference/api/rust-library-reference/nemo-relay-worker/enum-builtinllmcodec): Relay's built-in LLM codec identities.
* [DiagnosticLevel](/reference/api/rust-library-reference/nemo-relay-worker/enum-diagnosticlevel): Severity of a plugin configuration or activation diagnostic.
* [Event](/reference/api/rust-library-reference/nemo-relay-worker/enum-event): Tagged union covering the two ATOF event kinds emitted by the runtime.
* [LlmCodecIdentity](/reference/api/rust-library-reference/nemo-relay-worker/enum-llmcodecidentity): Per-call LLM codec identity supplied to sanitizer and SDK callbacks.
* [LlmOptimizationEvidenceQuality](/reference/api/rust-library-reference/nemo-relay-worker/enum-llmoptimizationevidencequality): Whether token evidence was observed directly or estimated.
* [LlmOptimizationSummaryStatus](/reference/api/rust-library-reference/nemo-relay-worker/enum-llmoptimizationsummarystatus): Completeness of close-time optimization accounting.
* [LogSeverity](/reference/api/rust-library-reference/nemo-relay-worker/enum-logseverity): Severity attached to a mark that may be projected as an OpenTelemetry log.
* [MetricKind](/reference/api/rust-library-reference/nemo-relay-worker/enum-metrickind): OpenTelemetry instrument kind represented by a metric measurement mark.
* [MetricValueType](/reference/api/rust-library-reference/nemo-relay-worker/enum-metricvaluetype): Numeric representation used by a metric measurement.
* [PluginLogLevel](/reference/api/rust-library-reference/nemo-relay-worker/enum-pluginloglevel): Process-log severity for records emitted through the Relay host.
* [RuntimeRegistrationKind](/reference/api/rust-library-reference/nemo-relay-worker/enum-runtimeregistrationkind): A global runtime registration surface that can be selected by a conditional middleware guardrail.
* [RuntimeRegistrationOwnerKind](/reference/api/rust-library-reference/nemo-relay-worker/enum-runtimeregistrationownerkind): The owner category reported for a global runtime registration.
* [ScopeType](/reference/api/rust-library-reference/nemo-relay-worker/enum-scopetype): Semantic category attached to a scope lifecycle span.
* [WorkerSdkError](/reference/api/rust-library-reference/nemo-relay-worker/enum-workersdkerror): Error returned by worker SDK callbacks and runtime helpers.

## Constants

* [ANNOTATED\_LLM\_REQUEST\_SCHEMA](/reference/api/rust-library-reference/nemo-relay-worker/constant-annotated-llm-request-schema): Versioned JSON envelope schema for [`AnnotatedLlmRequest`](/reference/api/rust-library-reference/nemo-relay-worker/struct-annotatedllmrequest).
* [METRIC\_DATA\_SCHEMA\_NAME](/reference/api/rust-library-reference/nemo-relay-worker/constant-metric-data-schema-name): Relay-owned schema name for mark payloads containing metric measurements.
* [METRIC\_DATA\_SCHEMA\_VERSION](/reference/api/rust-library-reference/nemo-relay-worker/constant-metric-data-schema-version): Current Relay-owned metric measurement schema version.

## Traits

* [LlmOptimizationPayload](/reference/api/rust-library-reference/nemo-relay-worker/trait-llmoptimizationpayload): Trait implemented by typed plugin payloads embedded in an optimization contribution.
* [WorkerPlugin](/reference/api/rust-library-reference/nemo-relay-worker/trait-workerplugin): Trait implemented by Rust out-of-process worker plugins.

## Functions

* [serve\_plugin](/reference/api/rust-library-reference/nemo-relay-worker/fn-serve-plugin): Serves a worker plugin using environment variables supplied by the Relay host.
* [serve\_plugin\_arc](/reference/api/rust-library-reference/nemo-relay-worker/fn-serve-plugin-arc): Serves a shared worker plugin using environment variables supplied by the Relay host.
* [serve\_plugin\_arc\_with\_config](/reference/api/rust-library-reference/nemo-relay-worker/fn-serve-plugin-arc-with-config): Serves a shared worker plugin using explicit endpoint and authentication configuration.

## Type Aliases

* [BoxFutureResult](/reference/api/rust-library-reference/nemo-relay-worker/type-boxfutureresult): Boxed future returned by async worker callbacks.
* [Json](/reference/api/rust-library-reference/nemo-relay-worker/type-json): Type alias for [`serde_json::Value`](https://docs.rs/serde_json/1.0.149/serde_json/value/enum.Value.html), used as the universal JSON representation throughout NeMo Relay APIs.
* [JsonStream](/reference/api/rust-library-reference/nemo-relay-worker/type-jsonstream): Boxed JSON stream returned by streaming worker callbacks.
* [LlmRequestContext](/reference/api/rust-library-reference/nemo-relay-worker/type-llmrequestcontext): Request codec context supplied to an LLM execution intercept.
* [LlmResponseContext](/reference/api/rust-library-reference/nemo-relay-worker/type-llmresponsecontext): Response codec context supplied to an LLM execution intercept.
* [Result](/reference/api/rust-library-reference/nemo-relay-worker/type-result): SDK result type.