About gRPC Worker Plugins

A worker plugin runs outside the Relay process and implements the stable grpc-v1 service contract. It can register the same 15 subscriber and middleware surfaces as an in-process plugin, call continuations back in the host, use invocation-scoped codecs, and emit marks or manage scope stacks through host-runtime RPCs.

Relay 0.8 establishes canonical tool results as the grpc-v1 baseline. Tool callbacks and ToolNext continuations preserve an application result and optional opaque annotation; an execution intercept can add Relay-owned pending marks. Rebuild every SDK worker, regenerate custom protobuf bindings, and declare compat.relay beginning at 0.8.0. The protocol remains named grpc-v1; this is a changed tool-result contract, not a new protocol family.

Workers can add a data schema and log severity to a mark, emit validated metric measurements, and read the current host-level runtime diagnostics. These diagnostics are ordered by code and are not attributed to a particular plugin.

Choose a worker when process or dependency isolation is worth the gRPC dispatch, JSON-envelope conversion, and scheduling cost on callback paths. The boundary limits the blast radius of a worker crash and keeps dependencies out of the application environment, but it is not a security sandbox. Relay authenticates a local endpoint with a per-activation token, and a worker is trusted to request host operations within that activation.

Choose the Implementation

Async Python and Rust callback work is cancelled cooperatively when the host caller times out, a stream consumer disconnects, or shutdown begins. Dropping a future or cancelling an asyncio task cannot preempt arbitrary blocking work or a separately spawned process. Keep blocking work off the SDK event loop, propagate cancellation, and put external cleanup in a guaranteed finalizer.

Let the SDK Own the Transport

Worker authors implement WorkerPlugin; they do not implement the protobuf server in application code. serve_plugin reads the activation environment created by Relay, binds the authenticated local endpoint, performs handshake and health handling, and dispatches validation, registration, invocation, cancellation, and shutdown.

Configure the Snapshot Location

Before starting a worker, Relay copies its runtime closure into an isolated activation snapshot. Set NEMO_RELAY_PLUGIN_SNAPSHOT_DIR in the environment that starts Relay to use a directory of your choice; Relay creates it when it does not exist. Choose a location outside every directory copied into a snapshot, including active plugin packages and external entrypoint or library directories. This applies to Python, Rust, and custom-command workers; it is particularly useful when the system temporary directory is too long or a deployment requires snapshots on controlled storage.

1
$env:NEMO_RELAY_PLUGIN_SNAPSHOT_DIR = "C:\\nrs"

$
export NEMO_RELAY_PLUGIN_SNAPSHOT_DIR=/var/tmp/nrs

1
import asyncio
2
from nemo_relay_plugin import WorkerPlugin, serve_plugin
3
4
class ExamplePythonWorker(WorkerPlugin):
5
    plugin_id = "examples.python_grpc_worker"
6
    allows_multiple_components = False
7
8
    def validate(self, config):
9
        return validate_config(config)
10
11
    def register(self, context, config):
12
        settings = normalized_config(config)
13
        if settings["observe"]["enabled"]:
14
            self._register_observation(
15
                context, settings["tag"], settings["observe"]
16
            )
17
        if settings["requests"]["enabled"]:
18
            self._register_requests(
19
                context,
20
                settings["tag"],
21
                settings["requests"],
22
                settings["execution"],
23
            )
24
        self._register_runtime(context, settings["tag"], settings["runtime"])
25
        if settings["execution"]["enabled"]:
26
            self._register_execution(context, settings["tag"], settings["execution"])
27
28
async def main():
29
    await serve_plugin(ExamplePythonWorker())
30
31
if __name__ == "__main__":
32
    asyncio.run(main())

Both register methods are synchronous because they describe and return the component’s registration set during activation. The callbacks they install can be asynchronous. The stable plugin_id must match the manifest ID and handshake identity; a mismatch causes activation to fail before any callback is routed.

Complete Path

Follow these pages in order to build, activate, exercise, and stop a worker plugin:

1. Use Python Worker when Relay should provision and own the Python environment, or Rust Worker for the checked compiled counterpart.
2. Implement all callback families with Middleware and Continuations, paying particular attention to repeated downstream calls and lazy streams.
3. Use Runtime Events and Scopes for marks, scope stacks, binding, restoration, and failure cleanup.
4. Use the grpc-v1 Protocol Reference to audit an SDK or implement a custom command.

The atomic Rust lifecycle test builds the executable in an isolated target directory, materializes and integrity-checks its manifest, completes worker activation, executes a managed tool call across grpc-v1, observes a host-runtime mark, and verifies orderly shutdown. The Rust and Python callback-contract tests then isolate the sanitizer, continuation, streaming, codec, cancellation, and scope-cleanup behavior. Merely starting a process and registering one intercept is not enough evidence.