Native Dynamic Plugins (Rust)

Native dynamic plugins are trusted in-process shared libraries. Use them when plugin behavior needs host-process access while still following the Relay plugin contract: a stable kind, JSON component configuration, validation diagnostics, and registration through a component-scoped context.

Manifest

Use the following manifest:

1
manifest_version = 1
2
3
[plugin]
4
id = "acme.native_policy"
5
kind = "rust_dynamic"
6
7
[compat]
8
relay = ">=0.5,<1.0"
9
native_api = "1"
10
11
[defaults]
12
enabled = false
13
14
[capabilities]
15
items = ["plugin_native"]
16
17
[source]
18
artifact = "target/release/libacme_native_policy.dylib"
19
20
[integrity]
21
sha256 = "sha256:<artifact-sha256>"
22
23
[load]
24
library = "target/release/libacme_native_policy.dylib"
25
symbol = "nemo_relay_register_plugin"

Set plugin.kind to rust_dynamic, compat.native_api to "1", and capabilities.items to include plugin_native. Relay resolves relative paths from the manifest. The exported symbol must return a descriptor whose plugin_kind matches plugin.id exactly.

If the plugin registers an LLM request intercept, set compat.relay = ">=0.6,<1.0", or another range that excludes Relay 0.5. Relay rejects the registration when the manifest admits a host that predates the version 2 request annotation envelope.

Create a Native Plugin

Create a Rust library project with the following Cargo.toml file:

1
[package]
2
name = "acme-native-policy"
3
version = "0.1.0"
4
edition = "2024"
5
6
[lib]
7
crate-type = ["cdylib"]
8
9
[dependencies]
10
nemo-relay-plugin = "0.5.0"
11
serde_json = "1"

Add the following implementation to src/lib.rs:

1
use nemo_relay_plugin::{Json, NativePlugin, PluginContext, Result};
2
use serde_json::Map;
3
4
struct NativePolicy;
5
6
impl NativePlugin for NativePolicy {
7
    fn plugin_kind(&self) -> &str {
8
        "acme.native_policy"
9
    }
10
11
    fn register(
12
        &mut self,
13
        _config: &Map<String, Json>,
14
        context: &mut PluginContext<'_>,
15
    ) -> Result<()> {
16
        context.register_subscriber("audit", |_| {})?;
17
        Ok(())
18
    }
19
}
20
21
nemo_relay_plugin::nemo_relay_plugin!(nemo_relay_register_plugin, || NativePolicy);

Build the library with the following command:

$
cargo build --release

Update source.artifact and load.library with the resulting platform library path, then replace <artifact-sha256> with that library’s SHA-256 digest. Use .dylib on macOS, .so on Linux, or .dll on Windows. Refer to Build a Rust Native Plugin for a complete example with validation, middleware, scopes, and configuration schema support.

Native ABI v3

The entry symbol receives a *const NemoRelayNativeHostApiV1 pointer. It points at the v1 prefix of a v3 NemoRelayNativeHostApiV3 table; check abi_version and struct_size before casting. The plugin returns a NemoRelayNativePluginV1 descriptor:

1
extern "C" fn nemo_relay_register_plugin(
2
    host: *const NemoRelayNativeHostApiV1,
3
    out: *mut NemoRelayNativePluginV1,
4
) -> NemoRelayStatus

The v3 host table retains the frozen legacy prefix and appends a completion-based asynchronous middleware extension. An entry that rejects the v3 table with InvalidArg is retried with the legacy table. Rust plugins using the typed NativePlugin APIs continue to work unchanged. Raw ABI plugins can use PluginContext::register_async_middleware_raw when a callback must complete later. The callback receives a JSON invocation, an optional continuation for execution intercepts, and a one-shot completion handle.

Return Complete after resolving or rejecting the completion before the callback returns. Return Pending only when retaining the completion; settle it exactly once, call async_completion_release, and release an async next handle after use. The host marks a completion cancelled when the awaiting runtime work is dropped; late and duplicate settlement is rejected safely. Invoke and finish every next operation before resolving or rejecting the owning completion. Relay rejects new calls and cancels unfinished calls after the completion settles.

For unary execution intercepts, use async_next_invoke_result to call next repeatedly or concurrently with an independent result callback for each call. The older completion-coupled async_next_invoke is one-shot because its result settles the middleware completion. For incremental stream intercepts, the output stream owns the callback lifetime: async_next_invoke_stream may run repeatedly or concurrently until the output stream finishes, rejects, or is cancelled. Each call requires independent callback state. Relay rejects later calls and cancels unfinished calls when the output stream settles.

Relay invokes an asynchronous middleware callback synchronously on the Tokio runtime worker that is polling that middleware invocation. There is no stable OS-thread affinity, and separate invocations can run concurrently. After returning Pending, a plugin can use its retained completion, stream, or next handle from another plugin-owned thread. The host synchronizes those opaque handles, schedules next on the captured Relay runtime, and can invoke an incremental downstream-stream callback on a Relay runtime worker. The plugin must synchronize shared user_data and callback state, and must keep them alive until the corresponding host-owned registration and all callback-owned handle references are released. Do not race a handle’s release operation against settlement, cancellation inspection, stream operations, or next invocation using that same reference; serialize the final release after the last such call returns.

Event sanitizers registered through this extension still run on Relay’s serial publication dispatcher. Scope and mark emission remain synchronous and their sanitized events are delivered later in emission order.

The generic v3 completion registration settles one JSON value and rejects the LlmStreamExecutionIntercept kind. Register asynchronous stream intercepts with plugin_context_register_async_stream_middleware instead. Its dedicated async_next_invoke_stream continuation forwards downstream chunks incrementally, so Relay does not buffer the provider stream into an array.

The incremental output queue is bounded. async_stream_push_json and async_stream_reject never block a native callback thread. If either returns Internal and the host last-error contains backpressured, retain the logical chunk or rejection and retry after the consumer advances. InvalidArg means the stream is already closed or cancelled and must not be retried. Check async_stream_is_cancelled during longer producer work.

For async_next_invoke_stream, Relay reports downstream failure or consumer cancellation through one terminal callback with a non-null error, and reports clean completion with done = true. Reclaim the callback’s user_data in that terminal callback. If a chunk callback returns false, reclaim user_data before returning because Relay does not invoke another callback afterward.

Cancelling the one-shot completion supplied to a non-stream execution intercept also aborts any pending async_next_invoke continuation. Plugins must still release their callback-owned completion and next references exactly once; cancellation only stops the host-side continuation and prevents it from retaining the plugin indefinitely.

Legacy v1/v2 middleware callbacks are synchronous and run on the runtime’s execution path. They must not block on I/O; use the v3 completion-based API for long-running work.

Text and JSON data cross this boundary as host-owned NemoRelayNativeString handles. ABI structs also carry scalars, opaque handles, callback pointers, and plugin-owned user_data. Do not pass Rust runtime types, trait objects, futures, serde_json::Value, or allocator-owned strings across the ABI.

ABI callbacks can register these runtime surfaces:

• Subscribers
• Tool and LLM guardrails or intercepts, including stream intercepts
• Marks, scopes, and isolated scope stacks

Relay keeps the library alive while those registrations exist and deregisters them before unloading it.

LLM sanitize callbacks receive their request or response JSON first, followed by NemoRelayNativeLlmSanitizeRequestContext or NemoRelayNativeLlmSanitizeResponseContext. Each context contains structured codec identity and a borrowed, callback-lifetime codec handle. codec_kind is None, BuiltIn, Runtime, or Opaque. codec_id is present for BuiltIn (one of openai_chat, openai_responses, or anthropic_messages) and Runtime, and null for None and Opaque.

The request handle supports host operations to decode an LlmRequest into an AnnotatedLlmRequest and encode normalized changes onto the original request. The response handle supports decoding response JSON into an AnnotatedLlmResponse. A null handle means no codec is active. Runtime and opaque codecs still have non-null handles and support the same operations as a built-in codec. Do not retain a handle or resolved SDK facade after the sanitizer callback returns.

The Rust SDK converts the raw structures into LlmSanitizeRequestContext and LlmSanitizeResponseContext. Call resolve_codec() to obtain the safe directional facade. Raw ABI consumers use the llm_request_codec_decode, llm_request_codec_encode, and llm_response_codec_decode host-table functions. The host owns all returned JSON string handles, which callers release with the ordinary host string release operation.

A successful null sanitizer output omits the LLM observability payload and its annotation. Returning an error also omits the payload and annotation; Relay records the callback error. Neither case changes the client-visible request or response.

Use the nemo-relay-plugin crate rather than the host nemo-relay runtime crate. Refer to Build a Rust Native Plugin for the SDK-backed example.

Register the Plugin

After you package the manifest and library, register and validate the plugin:

$
nemo-relay plugins add ./relay-plugin.toml
$
nemo-relay plugins enable acme.native_policy
$
nemo-relay plugins validate acme.native_policy

Refer to Configure Discoverable Plugins for lifecycle and trust-policy configuration.