nemo-relay-plugin

This page is for NeMo Relay, version v0.11.0. For other versions, use one of these documentation indexes: - Latest (v0.10.4) (default): https://docs.nvidia.com/nemo/relay/latest/llms.txt - dev: https://docs.nvidia.com/nemo/relay/dev/llms.txt - v0.11.0: https://docs.nvidia.com/nemo/relay/v0.11.0/llms.txt - v0.10.4: https://docs.nvidia.com/nemo/relay/v0.10.4/llms.txt - v0.9.4: https://docs.nvidia.com/nemo/relay/v0.9.4/llms.txt - v0.8.4: https://docs.nvidia.com/nemo/relay/v0.8.4/llms.txt - v0.7.3: https://docs.nvidia.com/nemo/relay/v0.7.3/llms.txt

Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-pii-redaction -p nemo-relay-ffi -p nemo-relay-types -p nemo-relay-plugin -p nemo-relay-worker-proto -p nemo-relay-worker.

Stable native plugin ABI and Rust authoring helpers for NeMo Relay.

This crate intentionally does not depend on the nemo-relay runtime crate. Native plugins built with it communicate with a host through versioned C-compatible tables and host-owned string handles.

Modules

• diagnostics: Native diagnostic authoring and authorized reads. Native diagnostic authoring and explicitly authorized canonical reads. Recording is synchronous, state-only, and independent of operational logs. Disabled/unsupported facades skip factories. Diagnostic errors must not be propagated as plugin execution failures unless the application chooses so.

Macros

• nemo_relay_plugin: Exports a concrete plugin constructor as a native plugin entry symbol body.
• relay_debug: Emit a formatted debug-level native-plugin record through the Relay host.
• relay_error: Emit a formatted error-level native-plugin record through the Relay host.
• relay_info: Emit a formatted native-plugin log record through the Relay host.
• relay_trace: Emit a formatted trace-level native-plugin record through the Relay host.
• relay_warn: Emit a formatted warn-level native-plugin record through the Relay host.

Structs

• AnnotatedLlmRequest: Structured view of an LLM request, produced by a Codec from opaque LlmRequest content.
• AnnotatedLlmResponse: Structured view of an LLM response, produced by a response codec from raw JSON API output.
• CategoryProfile: Category-specific profile data.
• ConditionalMiddlewareGuardrailHandle: Opaque activation-owned handle for a native plugin’s dynamic gate.
• ConfigDiagnostic: Structured diagnostic reported while validating or activating plugin configuration.
• DataSchema: Identifier for the schema that describes an event’s opaque data payload.
• EventCategory: Semantic category carried by ATOF category.
• EventSanitizeFields: Observability fields that event sanitizers may rewrite.
• LlmAttributes: Bitflags that modify LLM-call behavior and observability.
• LlmExecutionContext: Per-call codec context delivered to an LLM execution intercept.
• LlmExecutionRequestCodec: Invocation-lifetime request codec facade for an LLM execution intercept.
• LlmExecutionResponseCodec: Response codec for one non-streaming LLM execution intercept.
• LlmNext: Cloneable asynchronous LLM execution continuation.
• LlmOptimizationContribution: One optimizer’s evidence for a change to an LLM call.
• LlmOptimizationKind: Open, forward-compatible optimization classification.
• LlmOptimizationModel: Model identity used for counterfactual and effective pricing.
• LlmOptimizationModelTransition: A model change proposed or applied by an optimizer.
• LlmOptimizationSummary: Close-time accounting attached to the normalized LLM response.
• LlmOptimizationTokenImpact: Shared token evidence that Relay can aggregate without understanding a plugin payload.
• LlmOptimizationTokens: Token quantities retained independently from pricing arithmetic.
• LlmRequest: JSON-shaped LLM request payload passed through the runtime.
• LlmRequestContext: Request codec context for one LLM execution intercept invocation.
• LlmRequestInterceptOutcome: Result of an LLM request intercept that can schedule lifecycle marks.
• LlmResponseContext: Response codec context for one non-streaming LLM execution intercept invocation.
• LlmSanitizeRequestCodec: Safe completion-backed request codec facade for typed native plugins.
• LlmSanitizeRequestContext: Per-call request codec context delivered to an LLM sanitizer.
• LlmSanitizeResponseCodec: Safe completion-backed response codec facade for typed native plugins.
• LlmSanitizeResponseContext: Per-call response codec context delivered to an LLM sanitizer.
• LlmStream: Host- or plugin-owned stream returned across the native LLM stream ABI.
• LlmStreamNext: Cloneable asynchronous LLM stream continuation.
• MetricEnvelope: Payload stored in a mark using METRIC_DATA_SCHEMA_NAME.
• MetricMeasurement: One recording operation in a Relay metric mark.
• NativeExecutorConfig: Configuration for the executor owned by one exported native plugin.
• NemoRelayNativeAsyncCompletion: Opaque one-shot completion retained by a pending native callback.
• NemoRelayNativeAsyncNext: Opaque native execution continuation supplied only to execution intercepts.
• NemoRelayNativeAsyncStream: Opaque incremental output channel supplied to native async stream intercepts.
• NemoRelayNativeHostApiV1: Versioned host API table passed to native plugin entry symbols.
• NemoRelayNativeHostApiV3: ABI-v3 host extension appended to NemoRelayNativeHostApiV1.
• NemoRelayNativeHostApiV4: ABI-v4 host extension for typed asynchronous middleware, mark options, diagnostics, and activation-owned dynamic gate control.
• NemoRelayNativeHostApiV5: ABI-v5 host extension for context-aware raw tool execution intercepts.
• NemoRelayNativeHostApiV6: ABI-v6 host extension for operational logging.
• NemoRelayNativeHostApiV7: ABI-v7 host table for context-aware LLM execution intercept callbacks.
• NemoRelayNativeHostApiV8: ABI-v8 append-only scope inspection extension.
• NemoRelayNativeLlmAsyncStream: Opaque pull-based downstream LLM stream owned by the host.
• NemoRelayNativeLlmExecutionContext: Codec context passed to native LLM execution intercept callbacks.
• NemoRelayNativeLlmRequestCodec: Opaque host-owned request codec capability.
• NemoRelayNativeLlmRequestContext: Request codec context passed to a native LLM execution intercept.
• NemoRelayNativeLlmResponseCodec: Opaque callback-scoped response codec capability owned by the host.
• NemoRelayNativeLlmResponseContext: Response codec context passed to a native non-streaming LLM execution intercept.
• NemoRelayNativeLlmSanitizeRequestContext: Per-call LLM sanitizer context passed over the native ABI.
• NemoRelayNativeLlmSanitizeResponseContext: Per-call response sanitizer context passed over the native ABI.
• NemoRelayNativeLlmStreamV1: Native LLM JSON stream handle table.
• NemoRelayNativePluginContext: Opaque plugin registration context borrowed from the host during registration.
• NemoRelayNativePluginRuntime: Opaque activation-owned runtime capability used by native plugins.
• NemoRelayNativePluginV1: Versioned plugin descriptor returned by native plugin entry symbols.
• NemoRelayNativeScopeHandle: Opaque host-owned scope handle.
• NemoRelayNativeScopeStack: Opaque host-owned scope stack handle.
• NemoRelayNativeScopeStackBinding: Opaque host-owned captured scope-stack binding.
• NemoRelayNativeString: Opaque host-owned UTF-8 string or JSON byte buffer.
• PendingMarkSpec: Mark requested by middleware for materialization by a lifecycle owner.
• PluginContext: Borrowed safe wrapper around a host plugin registration context.
• PluginRuntime: Cloneable high-level runtime handle for host APIs available to native plugins.
• RuntimeDiagnostic: One bounded runtime diagnostic reported by the Relay host.
• RuntimeDiagnostics: Bounded snapshot of active host runtime diagnostics.
• RuntimeRegistrationIdentity: Structured identity for a global gateable runtime registration.
• RuntimeRegistrationOwner: Discovery metadata describing the owner of a runtime registration.
• ScopeAttributes: Bitflags that modify scope behavior and observability.
• ScopeGuard: RAII guard for a host scope opened by PluginRuntime::scope or PluginRuntime::scope_at.
• ScopeHandle: Host-owned scope handle returned by native scope APIs.
• ScopeStack: Host-owned isolated scope stack returned by native scope-stack APIs.
• ScopeStackBinding: Captured thread-local scope-stack binding.
• ScopeStackEntry: Identity of one entry in the visible Relay scope stack.
• ScopeStackSnapshot: Owned root-to-top values from one coherent read of a visible scope stack.
• ThreadScopeStackGuard: RAII guard that restores the previous thread-local scope stack on drop.
• ToolAttributes: Bitflags that modify tool-call behavior and observability.
• ToolExecutionInterceptOutcome: Canonical result returned by a tool execution intercept.
• ToolExecutionResult: Canonical application-visible result of tool execution.
• ToolNext: Cloneable asynchronous tool execution continuation.

Enums

• BuiltinLlmCodec: Relay’s built-in LLM codec identities.
• DiagnosticLevel: Severity of a plugin configuration or activation diagnostic.
• Event: Tagged union covering the two ATOF event kinds emitted by the runtime.
• HandleAttributes: Attribute bitflags attached to a concrete handle kind.
• LlmCodecIdentity: Per-call LLM codec identity supplied to sanitizer and SDK callbacks.
• LlmOptimizationEvidenceQuality: Whether token evidence was observed directly or estimated.
• LlmOptimizationSummaryStatus: Completeness of close-time optimization accounting.
• LogSeverity: Severity attached to a mark that may be projected as an OpenTelemetry log.
• MetricKind: OpenTelemetry instrument kind represented by a metric measurement mark.
• MetricValueType: Numeric representation used by a metric measurement.
• NemoRelayNativeAsyncCallbackState: Indicates whether an asynchronous native callback settled before returning.
• NemoRelayNativeAsyncMiddlewareKind: Middleware surface selected by the native async registration hook.
• NemoRelayNativeLlmCodecKind: Discriminator for the codec supplied to an LLM sanitizer over the native ABI.
• NemoRelayNativeLogLevel: Operational log severity accepted by the native host logging extension.
• NemoRelayNativeScopeType: Scope category used by native plugins when opening scopes.
• NemoRelayStatus: Status codes returned by stable native ABI functions.
• RuntimeRegistrationKind: A global runtime registration surface that can be selected by a conditional middleware guardrail.
• RuntimeRegistrationOwnerKind: The owner category reported for a global runtime registration.
• ScopeCategory: Agent Trajectory Observability Format (ATOF) lifecycle phase for a scope event.
• ScopeType: Semantic category attached to a scope lifecycle span.

Constants

• METRIC_DATA_SCHEMA_NAME: Relay-owned schema name for mark payloads containing metric measurements.
• METRIC_DATA_SCHEMA_VERSION: Current Relay-owned metric measurement schema version.
• NEMO_RELAY_NATIVE_ABI_VERSION: Native plugin ABI version supported by this crate.
• NEMO_RELAY_NATIVE_ABI_VERSION_ASYNC_MIDDLEWARE: ABI version that introduced completion-based asynchronous middleware.
• NEMO_RELAY_NATIVE_ABI_VERSION_LEGACY: Legacy native plugin ABI accepted by Relay hosts for compatibility.
• NEMO_RELAY_NATIVE_ABI_VERSION_LLM_EXECUTION_CONTEXT: ABI version that introduced uniform LLM execution codec context.
• NEMO_RELAY_NATIVE_ABI_VERSION_LOGGING: ABI version that introduced host-routed operational logging.
• NEMO_RELAY_NATIVE_ABI_VERSION_RUNTIME_CONTROL: ABI version that introduced runtime diagnostics and dynamic gate control.
• NEMO_RELAY_NATIVE_ABI_VERSION_SCOPE_STACK_SNAPSHOT: Append-only scope inspection extension, compatible with ABI v7 callbacks.
• NEMO_RELAY_NATIVE_ABI_VERSION_TOOL_EXECUTION_CONTEXT: ABI version that introduced context-aware raw tool execution intercepts.
• TOOL_EXECUTION_INTERCEPT_OUTCOME_SCHEMA: Versioned native-plugin JSON envelope schema for ToolExecutionInterceptOutcome.
• TOOL_EXECUTION_RESULT_SCHEMA: Versioned native-plugin JSON envelope schema for ToolExecutionResult.

Traits

• LlmOptimizationPayload: Trait implemented by typed plugin payloads embedded in an optimization contribution.
• NativePlugin: Trait implemented by Rust native plugins.

Functions

• capture_scope_stack_thread: Captures the current thread-local scope-stack binding.
• create_scope_stack: Creates a new independent scope stack.
• current_scope: Retrieves the current scope handle.
• emit_mark: Emits a mark event under the current scope.
• export_plugin: Initializes a native plugin descriptor for a Rust SDK plugin value.
• http_error_message: Formats an upstream HTTP error for transport through string-based plugin callbacks.
• pop_scope: Pops a scope and emits its end event.
• push_scope: Pushes a scope and emits its start event.

Type Aliases

• Json: Type alias for serde_json::Value, used as the universal JSON representation throughout NeMo Relay APIs.
• LlmJsonAsyncStream: Asynchronous JSON stream returned by a native typed stream interceptor.
• LlmJsonStream: Synchronous JSON chunk stream used by native LLM stream intercept helpers.
• NemoRelayNativeAsyncLlmExecutionCb: Completion-based native LLM execution callback.
• NemoRelayNativeAsyncLlmStreamOpenCb: Receives the result of asynchronously opening a downstream LLM stream.
• NemoRelayNativeAsyncLlmStreamPullCb: Receives one item from a pull-based downstream LLM stream.
• NemoRelayNativeAsyncMiddlewareCb: Completion-based native middleware callback.
• NemoRelayNativeAsyncNextResultCb: Receives one completion from a non-streaming execution-continuation invocation.
• NemoRelayNativeAsyncNextStreamCb: Receives one downstream stream item. chunk_json is non-null for a chunk, error is non-null for failure or consumer cancellation, and done marks clean completion. Unless the callback itself returns false, the host invokes one terminal callback so the plugin can reclaim user_data. Return false to cancel downstream production after the current callback; in that case, reclaim user_data before returning because no later callback is made.
• NemoRelayNativeAsyncStreamMiddlewareCb: Incremental native LLM stream intercept callback.
• NemoRelayNativeConditionalMiddlewareCb: Native callback deciding whether a matching runtime registration is eligible.
• NemoRelayNativeEmitMarkV2Fn: Extended native mark-emission function introduced in ABI v4.
• NemoRelayNativeEventSanitizeCb: Native event observability-field sanitizer callback.
• NemoRelayNativeEventSubscriberCb: Native event subscriber callback.
• NemoRelayNativeFreeFn: Optional destructor for user data captured by native callbacks.
• NemoRelayNativeGetRuntimeDiagnosticsFn: Reads the active host runtime-diagnostics snapshot as canonical JSON.
• NemoRelayNativeLlmConditionalCb: Native LLM conditional-execution callback.
• NemoRelayNativeLlmExecutionCb: Native LLM execution intercept callback.
• NemoRelayNativeLlmNextFn: Runtime-provided continuation for LLM execution intercepts.
• NemoRelayNativeLlmRequestInterceptCb: Native LLM request intercept callback.
• NemoRelayNativeLlmSanitizeRequestCb: Native LLM request sanitizer callback. Return a successful null output to omit the observability payload and annotation. request_json is borrowed, but may be written directly to out_request_json as a pass-through; the host releases an aliased input/output once. Any other non-null output must be host-allocated and transfers ownership to the host.
• NemoRelayNativeLlmSanitizeResponseCb: Native LLM response sanitizer callback. Return a successful null output to omit the observability payload and annotation. payload_json is borrowed, but may be written directly to out_json as a pass-through; the host releases an aliased input/output once. Any other non-null output must be host-allocated and transfers ownership to the host.
• NemoRelayNativeLlmStreamCancelFn: Optional native stream cancellation callback.
• NemoRelayNativeLlmStreamDropFn: Optional native stream destructor callback.
• NemoRelayNativeLlmStreamExecutionCb: Native LLM stream execution intercept callback.
• NemoRelayNativeLlmStreamNextFn: Runtime-provided continuation for LLM stream execution intercepts.
• NemoRelayNativeLlmStreamPollFn: Native stream poll callback.
• NemoRelayNativePluginDropFn: Native plugin drop callback.
• NemoRelayNativePluginEntry: Native entry symbol type loaded by the host.
• NemoRelayNativePluginRegisterFn: Native plugin registration callback.
• NemoRelayNativePluginValidateFn: Native plugin validation callback.
• NemoRelayNativeScopeStackSnapshotFn: Scope snapshot callback returning an owned serialized buffer.
• NemoRelayNativeToolConditionalCb: Native tool conditional-execution callback.
• NemoRelayNativeToolExecutionCb: Native tool execution intercept callback.
• NemoRelayNativeToolExecutionContextCb: Native tool execution intercept callback receiving canonical context JSON.
• NemoRelayNativeToolJsonCb: Native JSON transform callback for tool request/response sanitizers and tool request intercepts.
• NemoRelayNativeToolNextFn: Runtime-provided continuation for tool execution intercepts.
• NemoRelayNativeWithScopeStackCb: Native callback executed while a host scope stack is temporarily active.
• Result: Result type used by the Rust native plugin SDK.