> This page is for NeMo Relay, version dev.
> For other versions, use one of these documentation indexes:
> - Latest (v0.11.1) (default): https://docs.nvidia.com/nemo/relay/latest/llms.txt
> - dev: https://docs.nvidia.com/nemo/relay/dev/llms.txt
> - v0.11.1: https://docs.nvidia.com/nemo/relay/v0.11.1/llms.txt
> - v0.10.4: https://docs.nvidia.com/nemo/relay/v0.10.4/llms.txt
> - v0.9.4: https://docs.nvidia.com/nemo/relay/v0.9.4/llms.txt
> - v0.8.4: https://docs.nvidia.com/nemo/relay/v0.8.4/llms.txt
> - v0.7.3: https://docs.nvidia.com/nemo/relay/v0.7.3/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/relay/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/relay/_mcp/server.

# Module middleware

> Shared middleware callback, stream, and invocation-context contracts.

Generated from `cargo doc --no-deps -p nemo-relay-builtins -p nemo-relay-diagnostics -p nemo-relay-logging -p nemo-relay-plugin-host -p nemo-relay-codecs -p nemo-relay -p nemo-relay-runtime -p nemo-relay-model-pricing -p nemo-relay-resource-metrics -p nemo-relay-observability -p nemo-relay-adaptive -p nemo-relay-pii-redaction -p nemo-relay-ffi -p nemo-relay-types -p nemo-relay-plugin -p nemo-relay-worker-proto -p nemo-relay-worker`.

Shared middleware callback, stream, and invocation-context contracts.

## Modules

* [callbacks](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/callbacks): Middleware callback and stream contracts. Callback type aliases used by the runtime middleware pipeline.
* [llm_execution_context](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/llm_execution_context): Invocation-scoped codec context. Invocation-scoped codec context for LLM execution intercepts.

## Structs

* [LlmExecutionContext](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/struct-llmexecutioncontext): Codec access for one LLM execution.
* [LlmJsonStream](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/struct-llmjsonstream): Stream of JSON chunks produced by the managed streaming LLM pipeline.
* [LlmSanitizeRequestContext](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/struct-llmsanitizerequestcontext): Per-call codec identity and capability for an LLM request.
* [LlmSanitizeResponseContext](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/struct-llmsanitizeresponsecontext): Per-call codec identity and capability for an LLM response.
* [ToolExecutionContext](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/struct-toolexecutioncontext): Per-call context delivered to a tool execution intercept.

## Enums

* [BuiltinLlmCodec](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/enum-builtinllmcodec): Relay's built-in LLM codec identities.
* [LlmCodecIdentity](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/enum-llmcodecidentity): Per-call LLM codec identity supplied to sanitizer and SDK callbacks.
* [LlmCollector](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/enum-llmcollector): Collector used to accumulate streaming response chunks.
* [LlmFinalizer](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/enum-llmfinalizer): Finalizer used to synthesize the aggregate streaming response payload.

## Traits

* [LlmStreamInner](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/trait-llmstreaminner): Internal close-aware stream implementation.

## Type Aliases

* [ConditionalMiddlewareGuardrailFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-conditionalmiddlewareguardrailfn): Decide whether matching global runtime registrations remain eligible.
* [EventMetadataInjectorFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-eventmetadatainjectorfn): Add flat metadata attributes to a fully constructed event.
* [EventSanitizeFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-eventsanitizefn): Sanitize mutable observability fields on a fully constructed event.
* [EventSubscriberFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-eventsubscriberfn): Consume runtime lifecycle events after they are emitted.
* [LlmAsyncCollectorFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-llmasynccollectorfn): Collector that is awaited; see [`LlmCollector`](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/enum-llmcollector).
* [LlmAsyncFinalizerFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-llmasyncfinalizerfn): Finalizer that is awaited; see [`LlmFinalizer`](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/enum-llmfinalizer).
* [LlmBorrowedCollectorFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-llmborrowedcollectorfn): Collector that reads each provider JSON chunk without copying it.
* [LlmCollectorFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-llmcollectorfn): Per-chunk collector used by the streaming LLM runtime.
* [LlmCollectorFuture](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-llmcollectorfuture): Future returned by an asynchronous collector.
* [LlmConditionalFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-llmconditionalfn): Decide whether an LLM call is allowed to continue.
* [LlmExecutionFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-llmexecutionfn): Wrap or replace non-streaming LLM execution.
* [LlmExecutionNextFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-llmexecutionnextfn): Continuation type invoked by non-streaming LLM execution intercepts.
* [LlmFallibleFinalizerFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-llmfalliblefinalizerfn): Finalizer that can fail; see [`LlmFinalizer`](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/enum-llmfinalizer).
* [LlmFinalizerFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-llmfinalizerfn): Finalizer used to synthesize the aggregate streaming response payload.
* [LlmFinalizerFuture](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-llmfinalizerfuture): Future returned by an asynchronous finalizer.
* [LlmRequestContext](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-llmrequestcontext): Request codec context exposed to an LLM execution intercept.
* [LlmRequestInterceptFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-llmrequestinterceptfn): Rewrite or annotate an LLM request before execution.
* [LlmResponseContext](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-llmresponsecontext): Response codec context exposed to a non-streaming LLM execution intercept.
* [LlmSanitizeRequestFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-llmsanitizerequestfn): Sanitize an LLM request before the runtime records it.
* [LlmSanitizeResponseFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-llmsanitizeresponsefn): Sanitize an LLM response before the runtime records it.
* [LlmStreamExecutionFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-llmstreamexecutionfn): Wrap or replace streaming LLM execution.
* [LlmStreamExecutionNextFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-llmstreamexecutionnextfn): Continuation type invoked by streaming LLM execution intercepts.
* [ToolConditionalFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-toolconditionalfn): Decide whether a tool call is allowed to continue.
* [ToolExecutionFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-toolexecutionfn): Wrap or replace tool execution with access to the full call context.
* [ToolExecutionNextFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-toolexecutionnextfn): Continuation type invoked by tool execution intercepts.
* [ToolInterceptFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-toolinterceptfn): Rewrite tool arguments before execution.
* [ToolSanitizeFn](/nemo/relay/dev/reference/api/rust-library-reference/nemo-relay/middleware/type-toolsanitizefn): Sanitize a tool request payload before the runtime records it.