About Native Dynamic Plugins

A native dynamic plugin is a trusted Rust shared library that Relay loads into its own process. It follows the same validation and PluginContext contract as other plugins, but typed middleware crosses a stable C boundary without a worker process or gRPC JSON envelope. This model is appropriate when reusable callback behavior is sensitive to latency or throughput and the deployment can carry a platform-specific binary.

Version Contracts

Three version values answer different questions:

The checked example registers a tool execution intercept, so it declares compat.relay = ">=0.10.0,<1.0". Every native plugin built with this SDK must use that lower bound because an older Relay cannot load the ABI v7 function table, even when the component itself does not register an LLM execution intercept.

Relay 0.10 adds ABI v7 with request and response codec context on every LLM execution callback. Rebuild all native plugins for this release; the authored native_api = "1" label does not change.

Relay 0.8 changed the native API 1 tool-result JSON contract without changing the v4 host-table layout. A tool callback and ToolNext continuation return ToolExecutionResult, which carries result and optional opaque annotation; an execution intercept returns that pair plus pending marks. Rebuild native plugins for that contract even though its negotiated host-table ABI remained v4. Relay 0.9 adds the context-carrying tool execution callback as the v5 host-table extension.

In Relay 0.8, the v4 SDK also let a plugin add a data schema and log severity to a mark, or emit validated metric measurements. PluginRuntime::runtime_diagnostics() returns the current host-level diagnostic snapshot. The snapshot is ordered by code and is not attributed to a particular plugin.

What the SDK Owns

The Rust SDK exports the stable entry symbol, converts host-owned JSON handles into typed DTOs, registers all 16 plugin surfaces, and drives async middleware on one SDK-owned multi-thread Tokio runtime per configured component. A plugin can set a default executor size and accept a positive executor.worker_threads component override. The default is two workers; change it only after measuring queued async work and account for the number of native components in the host.

Subscribers remain synchronous and run on Relay’s subscriber dispatcher. Typed middleware returns futures and runs on the SDK executor. A callback has no stable OS thread affinity, separate invocations can overlap, and blocking an executor thread can delay unrelated calls from the same component.

The checked-in examples/rust-native-plugin project is the end-to-end implementation. Its configuration, observation, request policy, execution wrappers, and runtime helpers are separated into modules so each following page can explain one responsibility without presenting a monolithic sample.

Implement the Native Plugin Entrypoint

The root module is intentionally small. validate checks both the example settings and the SDK-owned executor override. register parses the same settings once, obtains the component runtime handle, and delegates each feature group. The export macro produces the nemo_relay_register_plugin symbol named in the manifest.

mod config;
mod execution;
mod observe;
mod requests;
mod runtime;
use nemo_relay_plugin::{
    ConfigDiagnostic, DiagnosticLevel, Json, NativeExecutorConfig,
    NativePlugin, PluginContext,
};
use serde_json::Map;
struct ExampleNativePlugin;
impl NativePlugin for ExampleNativePlugin {
    fn plugin_kind(&self) -> &str {
        "examples.rust_native_policy"
    }
    fn executor_config(&self) -> NativeExecutorConfig {
        NativeExecutorConfig { worker_threads: 2 }
    }
    fn allows_multiple_components(&self) -> bool {
        false
    }
    fn validate(&self, plugin_config: &Map<String, Json>) -> Vec<ConfigDiagnostic> {
        let mut diagnostics = config::validate(plugin_config);
        if let Err(message) = self.executor_config_for_component(plugin_config) {
            diagnostics.push(ConfigDiagnostic {
                level: DiagnosticLevel::Error,
                code: "examples.rust_native_policy.invalid_executor".into(),
                component: Some("examples.rust_native_policy".into()),
                field: Some("executor.worker_threads".into()),
                message,
            });
        }
        diagnostics
    }
    fn register(
        &mut self,
        plugin_config: &Map<String, Json>,
        context: &mut PluginContext<'_>,
    ) -> nemo_relay_plugin::Result<()> {
        let config = config::ExampleConfig::parse(plugin_config)?;
        let runtime = context.runtime();
        observe::register(context, &config, &runtime)?;
        requests::register(context, &config)?;
        execution::register(context, &config, &runtime)?;
        Ok(())
    }
}
nemo_relay_plugin::nemo_relay_plugin!(
    nemo_relay_register_plugin,
    || ExampleNativePlugin
);

The kind in plugin_kind() must match [plugin].id in relay-plugin.toml. Returning false from allows_multiple_components tells Relay to reject a document that tries to activate two configurations of this native implementation. The SDK owns registrations created through context; the plugin does not keep raw registry handles or unload them itself.

Complete Path

Follow these pages in order to build, activate, exercise, and remove the native plugin:

1. Follow Build and Package to build the cdylib, validate its schema and manifest, calculate integrity, and register it.
2. Add observability behavior with Observe and Sanitize, including subscribers and all three event sanitizer surfaces.
3. Add policy and request rewriting with Control Requests, preserving annotations and making priority and break_chain explicit.
4. Add tool, unary model, and lazy stream wrappers with Wrap Execution.
5. Verify marks, scopes, isolated stacks, cleanup, and executor control with Runtime Events and Scopes.
6. Consult Native ABI Reference only when implementing or auditing the raw boundary.

Success is not merely a library that loads. The atomic lifecycle test builds the cdylib in an isolated target directory, materializes and integrity-checks its manifest, activates a valid component, executes a managed tool call, observes the runtime mark, and clears registrations before the library unloads. The focused configuration tests cover rejected input and schema shape; the scenario pages show the remaining callback contracts.

Forward HTTP error statuses

An execution callback can return an error string beginning with HTTP <status>: <message> to preserve an upstream HTTP error status through Relay. The prefix must appear at the start, use exactly three ASCII digits from 400 through 599, and include a space after the colon. Other error strings keep their existing handling.

Relay converts a recognized prefix to FlowError::Upstream with classification Other, an empty header map, and the message after the prefix. This convention does not enable retries. Use the final provider failure after routing and fallback attempts, and redact credentials and sensitive payloads before returning the message.

Recognized errors survive execution continuations and stream callbacks. Errors after response headers have been sent remain stream errors and cannot change the HTTP response status. No native ABI or worker protobuf change is required; older hosts treat the prefix as an ordinary error string.

The Rust native SDK exports http_error_message, which validates the status and returns the formatted string:

let message = nemo_relay_plugin::http_error_message(403, "Provider denied access")?;
return Err(message);

Existing plugins can also return a literal such as HTTP 404: Model not found. Returning a downstream next.call(...).await error unchanged preserves the prefix. Wrapping it with additional text before HTTP prevents recognition.

The native async streaming SDK opens its output stream before polling the callback. A callback failure is therefore delivered as a typed stream error; it does not change an HTTP status that the gateway has already sent.

Relay adds Reported by plugin '<plugin-id>': to recognized HTTP error messages using the plugin identity verified by the host. This identifies the reporting component; a provider or another downstream component may have produced the original failure. Reporting context survives continuations, without repeating the same leading plugin identity. Structured process logs include plugin_id, the native or worker boundary, and the HTTP status.