gRPC Worker Protocol Overview

grpc-v1 is the stable out-of-process plugin protocol for plugin.kind = "worker". Python, Rust, and other local executables can implement the nemo.relay.worker.v1 API. Relay starts every worker and connects to it through local endpoints. Remote worker endpoints are not supported.

Relay 0.8 resets the grpc-v1 tool-result contract to use structured protobuf ToolExecutionResult and ToolExecutionInterceptOutcome messages. Rebuild every worker and regenerate custom protobuf bindings for Relay 0.8. Declare a compat.relay range that excludes earlier releases. The recommended range is >=0.8.0,<1.0; open-ended or narrower 0.8-or-newer ranges are valid. The manifest is an author compatibility assertion, not artifact attestation.

Use the Rust or Python SDK unless you need another runtime. Refer to gRPC Worker Plugin Concepts for runtime choices and lifecycle guidance.

Service Contract

Workers implement the PluginWorker service:

• Handshake and Health identify a ready worker.
• Validate returns configuration diagnostics.
• Register returns declarative subscriber, guardrail, and intercept registrations.
• Invoke and InvokeStream run registered behavior.
• CancelInvocation requests cancellation, and Shutdown requests process termination.

Relay implements RelayHostRuntime for worker-initiated operations. It lets a worker emit marks, manage scopes and isolated scope stacks, and call tool, LLM, or LLM-stream continuations during execution intercepts.

This page defines the application contract. Refer to the canonical protobuf transport schema for every message and RPC field required to implement another runtime. The protobuf package, service names, and RPC method names remain at v1. The ToolNext response type and the ToolExecutionInterceptResult.outcome field type change for the Relay 0.8 baseline. Workers built against an earlier grpc-v1 schema are incompatible and must regenerate their bindings.

PluginWorker RPCs

The worker implements the following RPCs:

Every request carries the activation ID and authentication token. Validate and Register also carry the plugin ID and component configuration.

Registration and Invocation

On success, RegisterResponse contains Registration records with a local name, surface, priority, and break_chain value. A failed registration can return WorkerError without registrations. The supported surfaces are:

• SUBSCRIBER
• TOOL_SANITIZE_REQUEST_GUARDRAIL, TOOL_SANITIZE_RESPONSE_GUARDRAIL, TOOL_CONDITIONAL_EXECUTION_GUARDRAIL, TOOL_REQUEST_INTERCEPT, and TOOL_EXECUTION_INTERCEPT
• LLM_SANITIZE_REQUEST_GUARDRAIL, LLM_SANITIZE_RESPONSE_GUARDRAIL, LLM_CONDITIONAL_EXECUTION_GUARDRAIL, LLM_REQUEST_INTERCEPT, LLM_EXECUTION_INTERCEPT, and LLM_STREAM_EXECUTION_INTERCEPT

InvokeRequest identifies the registration, surface, invocation, optional continuation, and scope context. Its payload is one of an event, tool invocation, or LLM invocation. InvokeResponse returns an empty result, JSON result, guardrail result, LLM request-intercept result, tool-execution result, or WorkerError. InvokeStream emits JSON chunks or WorkerError chunks.

Every LLM sanitizer invocation includes a directional context with tagged codec identity: none, builtin(id), runtime(id), or opaque. Worker SDKs expose context.resolve_codec() for active codecs. The resulting invocation-scoped proxy supports request decode/encode or response decode and calls Relay through the host runtime; the capability identifier is protocol-internal and expires when the callback completes. resolve_codec() returns no proxy only when no codec is active. Runtime and opaque codecs remain resolvable.

Request and response sanitizer handlers always receive (payload, context). Return the sanitized payload to continue the chain, or return no payload to omit the observability payload and annotation without changing the client-visible value. Rust worker sanitizer callbacks are async for every sanitizer surface: mark, scope, tool, and LLM. Python worker sanitizers can return either an immediate value or an awaitable.

1
ctx.register_llm_sanitize_request_guardrail(
2
    "normalize-request",
3
    10,
4
    |request, context| async move {
5
        let Some(codec) = context.resolve_codec() else {
6
            return Ok(Some(request));
7
        };
8
        let mut annotated = codec.decode(&request).await?;
9
        annotated.messages.clear();
10
        Ok(Some(codec.encode(&annotated, &request).await?))
11
    },
12
);

RelayHostRuntime RPCs

Relay implements these RPCs for worker callbacks:

Every host-runtime request also carries the activation ID and authentication token. Scope operations include a ScopeContext; continuation calls include the continuation ID that Relay supplied for the active intercept. Codec operations additionally require the unforgeable capability ID supplied for the current sanitizer invocation. Relay rejects missing, forged, expired, wrong-direction, or activation-mismatched capabilities. Codec transformation failures are non-retryable worker errors.

The following protobuf types are normative for canonical tool results:

ToolExecutionResult contains a required result and optional annotation. ToolExecutionInterceptOutcome adds Relay-owned pending_marks as one JSON array. Each opaque JSON value uses JsonValue, whose bytes contain exactly one JSON value. This preserves arbitrary JSON and integer precision while generated protobuf clients enforce the surrounding result and annotation structure.

These are deliberate protobuf type changes under the Relay 0.8 grpc-v1 baseline. The package and RPC method names remain v1, but workers must regenerate their protobuf bindings and rebuild. Future incompatible worker protobuf or payload semantic changes must increment worker_protocol.

A worker may call an execution continuation repeatedly or concurrently while the corresponding Invoke or InvokeStream callback is active. Each call gets an isolated scope-stack branch containing the scopes visible at invocation. The worker must finish continuation calls before returning its middleware result; Relay removes the continuation ID and cancels unfinished calls when the callback settles. For InvokeStream, the returned worker stream extends the active callback lifetime until it closes, so it can call LlmStreamNext lazily. A stream successfully returned by LlmStreamNext keeps its normal streaming lifetime.

Authentication and Endpoints

Relay supplies an activation ID, an activation token, the worker endpoint, and the host endpoint when it starts the process. SDKs attach the activation ID and token to protocol calls. The host rejects requests with an invalid activation ID or token.

On Unix platforms, Relay uses local Unix sockets. On other platforms, Relay uses loopback TCP endpoints. Workers must not accept arbitrary remote endpoint configuration.

Payloads

Relay data values use this envelope:

1
message JsonEnvelope {
2
  string schema = 1;
3
  bytes json = 2;
4
}

Use JsonEnvelope for open Relay DTOs that the protocol does not model directly. Canonical tool results are the exception: protobuf defines their wrapper, and JsonValue carries the application result, annotation, and pending-mark array losslessly.

Activation and Shutdown

1. Run nemo-relay plugins validate <plugin-id> before enabling or running a plugin to check its manifest, trust evidence, and optional static configuration schema.
2. Relay creates local endpoints, starts the worker, and completes health and handshake checks.
3. Relay sends component configuration to Validate. Error diagnostics stop initialization after the worker process starts.
4. Relay obtains registrations from Register and installs proxy callbacks. Relay rolls back installed callbacks if initialization fails.
5. Relay removes proxy callbacks before it sends Shutdown to the worker.

Errors

gRPC status communicates transport, authentication, malformed protocol requests, and some stream failures. Registration and unary callback failures can return structured WorkerError values. A stream callback failure can terminate the gRPC stream with a status error. Worker implementations must handle both error forms.