Module callbacks

This page is for NeMo Relay, version v0.11.1. For other versions, use one of these documentation indexes: - Latest (v0.10.4) (default): https://docs.nvidia.com/nemo/relay/latest/llms.txt - dev: https://docs.nvidia.com/nemo/relay/dev/llms.txt - v0.11.1: https://docs.nvidia.com/nemo/relay/v0.11.1/llms.txt - v0.10.4: https://docs.nvidia.com/nemo/relay/v0.10.4/llms.txt - v0.9.4: https://docs.nvidia.com/nemo/relay/v0.9.4/llms.txt - v0.8.4: https://docs.nvidia.com/nemo/relay/v0.8.4/llms.txt - v0.7.3: https://docs.nvidia.com/nemo/relay/v0.7.3/llms.txt

Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-pii-redaction -p nemo-relay-ffi -p nemo-relay-types -p nemo-relay-plugin -p nemo-relay-worker-proto -p nemo-relay-worker.

Callback type aliases used by the runtime middleware pipeline.

The public middleware registration APIs accept callback closures with the signatures defined in this module. These aliases centralize those signatures so the runtime can compose tool and LLM middleware consistently across bindings.

Structs

• LlmJsonStream: Stream of JSON chunks produced by the managed streaming LLM pipeline.
• LlmSanitizeRequestContext: Per-call codec identity and capability for an LLM request.
• LlmSanitizeResponseContext: Per-call codec identity and capability for an LLM response.
• ToolExecutionContext: Per-call context delivered to a tool execution intercept.

Enums

• BuiltinLlmCodec: Relay’s built-in LLM codec identities.
• LlmCodecIdentity: Per-call LLM codec identity supplied to sanitizer and SDK callbacks.
• LlmCollector: Collector used to accumulate streaming response chunks.
• LlmFinalizer: Finalizer used to synthesize the aggregate streaming response payload.

Traits

• LlmStreamInner: Internal close-aware stream implementation.

Type Aliases

• ConditionalMiddlewareGuardrailFn: Decide whether matching global runtime registrations remain eligible.
• EventMetadataInjectorFn: Add flat metadata attributes to a fully constructed event.
• EventSanitizeFn: Sanitize mutable observability fields on a fully constructed event.
• EventSubscriberFn: Consume runtime lifecycle events after they are emitted.
• LlmAsyncCollectorFn: Collector that is awaited; see LlmCollector.
• LlmAsyncFinalizerFn: Finalizer that is awaited; see LlmFinalizer.
• LlmBorrowedCollectorFn: Collector that reads each provider JSON chunk without copying it.
• LlmCollectorFn: Per-chunk collector used by the streaming LLM runtime.
• LlmCollectorFuture: Future returned by an asynchronous collector.
• LlmConditionalFn: Decide whether an LLM call is allowed to continue.
• LlmExecutionFn: Wrap or replace non-streaming LLM execution.
• LlmExecutionNextFn: Continuation type invoked by non-streaming LLM execution intercepts.
• LlmFallibleFinalizerFn: Finalizer that can fail; see LlmFinalizer.
• LlmFinalizerFn: Finalizer used to synthesize the aggregate streaming response payload.
• LlmFinalizerFuture: Future returned by an asynchronous finalizer.
• LlmRequestContext: Request codec context exposed to an LLM execution intercept.
• LlmRequestInterceptFn: Rewrite or annotate an LLM request before execution.
• LlmResponseContext: Response codec context exposed to a non-streaming LLM execution intercept.
• LlmSanitizeRequestFn: Sanitize an LLM request before the runtime records it.
• LlmSanitizeResponseFn: Sanitize an LLM response before the runtime records it.
• LlmStreamExecutionFn: Wrap or replace streaming LLM execution.
• LlmStreamExecutionNextFn: Continuation type invoked by streaming LLM execution intercepts.
• ToolConditionalFn: Decide whether a tool call is allowed to continue.
• ToolExecutionFn: Wrap or replace tool execution with access to the full call context.
• ToolExecutionNextFn: Continuation type invoked by tool execution intercepts.
• ToolInterceptFn: Rewrite tool arguments before execution.
• ToolSanitizeFn: Sanitize a tool request payload before the runtime records it.