grpc-v1 Protocol Reference

grpc-v1 is the stable worker protocol implemented by the Rust and Python SDKs. The current source of truth is crates/worker-proto/proto/nemo/relay/worker/v1/plugin_worker.proto. Use SDK helpers to manage the lifecycle. The diagnostics helper accepts generated observation and code-definition messages directly.

Diagnostic Delivery

The optional RelayDiagnostics service shares the authenticated host endpoint. It exposes four RPCs:

• GetDiagnosticCapabilities.
• OpenDiagnosticSession.
• SubmitDiagnostic.
• ReadDiagnosticPage.

Its decode and admission limits are separate from RelayHostRuntime, logging, and execution payload limits.

Older hosts return UNIMPLEMENTED. For unsupported hosts or disabled collection, the SDK returns an inactive helper that skips lazy observation factories. Diagnostic availability does not decide whether a plugin executes.

Negotiate Settings and Authority

The host returns DiagnosticCapabilities before opening a session and again when the session opens. Both values must agree. The SDK checks contract versions and its local safety ceilings. It then uses the host’s exact request and response limits, queue capacity, retry count, and per-attempt timeout.

A mismatch or unsupported limit produces an inactive unsupported helper. Plugin execution continues. The SDK does not negotiate different limits or use local delivery defaults. Bootstrap calls have a separate local safety timeout. Flush and shutdown drain also have local bounds. These are separate from the host’s per-attempt RPC timeout.

Recording uses activation credentials. Those credentials do not grant snapshot read access: can_read_context is false, and canonical reads are denied. Query filters do not grant permission. The read RPC reserves a separate access boundary.

Opening an already live session returns the same session. The host assigns its UUID and ownership. Registration returns component and code handles. Code handles are one-based positions in the registration’s code list.

Workers can register common plugin failure codes, but not host-owned lifecycle codes. Extensions can describe failure, success, or unspecified outcomes under a host-assigned session namespace. Unknown authoring codes and failure classes are rejected. Detail and correlation limits come from the core context.

Submit and Retry Observations

SubmitDiagnostic uses one sequence, starting at one, for registration, recording, component close, and session close. Positive counts of two and three add five occurrences. The host commits a digest and receipt atomically with the record.

Receipts and sequence handling follow these rules:

• Retrying the last request with identical content returns its receipt without counting the observation again. Conflicts, gaps, and older sequences fail.
• Malformed requests do not consume a sequence. A structurally valid request rejected during admission consumes a sequence and returns a terminal Rejected receipt.
• Recorded confirms in-memory counts. Evidence can be retained, truncated, or dropped. Registered, Closed, and Disabled add no occurrences.

The SDK freezes request bytes and sends one request at a time. Each retry uses identical bytes, up to the host’s attempt count and per-attempt deadline. If the SDK cannot confirm acceptance, it stops delivery. Pending observations are never replayed into a new session.

This release allows one session per activation. A closed session keeps its last receipt until activation cleanup. Starting a new session requires a new activation.

Delivery Limits

Delivery uses the following limits in addition to core storage limits:

The service takes a concurrency permit before decoding protobuf. A slow response keeps that permit until its body reaches its delivery deadline or is dropped. HTTP/2 buffers, channel state, allocator overhead, and core pools have separate costs. These limits do not cap whole-process RSS.

Handle Loss and Shutdown

delivery_stats reports local dropped, rejected, and uncertain totals separately from host occurrence counts. An explicit INVALID_ARGUMENT response counts as rejected, even when the host leaves its sequence available for a corrected request.

A host capacity rejection uses RESOURCE_EXHAUSTED with nemo-relay-diagnostic-not-consumed: 1 status metadata. The SDK counts a definite rejection and preserves the sequence for later submissions. The metadata confirms only that attempt. If an earlier attempt had an uncertain result, the SDK keeps retrying the identical request within its policy. Without a receipt, delivery stays uncertain and stops. A client-side or unmarked capacity error is also uncertain. These decisions never depend on error-message text.

flush waits for earlier queued work. It returns an error for uncertain delivery. Counters expose known losses. close drains and releases queued work within the shutdown allowance. Activation teardown stops submissions and closes registrations. The runtime context remains available.

Restricted evidence uses state-only recording. It never enters Log, EmitMark, legacy reports, or exporters.

Execution Protocol Compatibility

Relay 0.8 retains the grpc-v1 identifier and nemo.relay.worker.v1 package, but it changes the tool-result boundary to structural protobuf messages. Rebuild workers, regenerate custom bindings, and declare compat.relay beginning at 0.8.0; an earlier worker cannot decode the current ToolNext response or tool-execution outcome.

Relay 0.10 retains those protocol identifiers and adds execution codec context to LlmInvocation. The field is protobuf-compatible, but the 0.10 Rust and Python SDKs change non-streaming and streaming LLM execution callbacks to receive LlmExecutionContext before the continuation. Workers that register either surface must update their callback signatures, rebuild, and declare compat.relay beginning at 0.10.0. Custom workers that register either surface must regenerate their bindings to read the field.

The protocol consists of the worker-facing service implemented by the plugin process and the host-runtime service implemented by Relay. These are the current service definitions, including cancellation, codecs, and streaming continuations:

service PluginWorker {
  rpc Handshake(HandshakeRequest) returns (HandshakeResponse);
  rpc Health(HealthRequest) returns (HealthResponse);
  rpc Validate(ValidateRequest) returns (ValidateResponse);
  rpc Register(RegisterRequest) returns (RegisterResponse);
  rpc Invoke(InvokeRequest) returns (InvokeResponse);
  rpc InvokeStream(InvokeRequest) returns (stream StreamChunk);
  rpc CancelInvocation(CancelInvocationRequest) returns (WorkerAck);
  rpc Shutdown(ShutdownRequest) returns (WorkerAck);
}
service RelayHostRuntime {
  rpc Log(LogRequest) returns (HostAck);
  rpc EmitMark(EmitMarkRequest) returns (HostAck);
  rpc GetCurrentScopeStackSnapshot(GetCurrentScopeStackSnapshotRequest)
      returns (GetCurrentScopeStackSnapshotResponse);
  rpc GetRuntimeDiagnostics(GetRuntimeDiagnosticsRequest)
      returns (GetRuntimeDiagnosticsResponse);
  rpc PushScope(PushScopeRequest) returns (PushScopeResponse);
  rpc PopScope(PopScopeRequest) returns (HostAck);
  rpc CreateScopeStack(CreateScopeStackRequest)
      returns (CreateScopeStackResponse);
  rpc DropScopeStack(DropScopeStackRequest) returns (HostAck);
  rpc ToolNext(ToolNextRequest) returns (ToolExecutionResultResponse);
  rpc LlmNext(LlmNextRequest) returns (JsonResult);
  rpc LlmStreamNext(LlmStreamNextRequest) returns (stream StreamChunk);
  rpc DecodeLlmCodecRequest(LlmCodecDecodeRequest) returns (JsonResult);
  rpc EncodeLlmCodecRequest(LlmCodecEncodeRequest) returns (JsonResult);
  rpc DecodeLlmCodecResponse(LlmCodecDecodeResponse) returns (JsonResult);
}

Worker Service

Each lifecycle message carries the following fields. Fields described as envelopes use the schema identifiers in the JSON envelope table later on this page.

Registration Surfaces

The numeric values are part of the wire contract. Unknown value zero never represents a valid registration.

enum RegistrationSurface {
  REGISTRATION_SURFACE_UNSPECIFIED = 0;
  SUBSCRIBER = 1;
  EVENT_METADATA_INJECTOR = 2;
  TOOL_SANITIZE_REQUEST_GUARDRAIL = 10;
  TOOL_SANITIZE_RESPONSE_GUARDRAIL = 11;
  TOOL_CONDITIONAL_EXECUTION_GUARDRAIL = 12;
  TOOL_REQUEST_INTERCEPT = 13;
  TOOL_EXECUTION_INTERCEPT = 14;
  LLM_SANITIZE_REQUEST_GUARDRAIL = 20;
  LLM_SANITIZE_RESPONSE_GUARDRAIL = 21;
  LLM_CONDITIONAL_EXECUTION_GUARDRAIL = 22;
  LLM_REQUEST_INTERCEPT = 23;
  LLM_EXECUTION_INTERCEPT = 24;
  LLM_STREAM_EXECUTION_INTERCEPT = 25;
  MARK_SANITIZE_GUARDRAIL = 30;
  SCOPE_SANITIZE_START_GUARDRAIL = 31;
  SCOPE_SANITIZE_END_GUARDRAIL = 32;
  CONDITIONAL_MIDDLEWARE_GUARDRAIL = 40;
}

An invocation names the activation, invocation, registration, surface, optional continuation, captured scope, and token. Its payload is exactly one event, tool invocation, LLM invocation, or conditional middleware invocation. LLM sanitizer invocations additionally carry codec identity and an opaque invocation-scoped codec capability. LLM execution invocations carry separate request and completed-response codec context. Streaming execution carries only the request context.

For EVENT_METADATA_INJECTOR, Relay sends the immutable Event snapshot in InvokeRequest.event. The worker returns an InvokeResponse.json object containing proposed additions to Event.metadata. Relay validates and merges accepted additions before Event sanitizers run. A WorkerError omits that callback’s additions without dropping the Event.

message InvokeRequest {
  string activation_id = 1;
  string invocation_id = 2;
  string registration_name = 3;
  RegistrationSurface surface = 4;
  string continuation_id = 5;
  ScopeContext scope = 6;
  string auth_token = 7;
  oneof payload {
    JsonEnvelope event = 10;
    ToolInvocation tool = 11;
    LlmInvocation llm = 12;
    ConditionalMiddlewareInvocation conditional_middleware = 13;
  }
}
message ConditionalMiddlewareInvocation {
  repeated RegistrationSurface kinds = 1;
  string registration_name = 2;
}
message LlmInvocation {
  string model_name = 1;
  JsonEnvelope request = 2;
  JsonEnvelope annotated_request = 3;
  JsonEnvelope response = 4;
  reserved 5, 6, 7, 8;
  oneof sanitize_context {
    LlmSanitizeRequestContext request_sanitize_context = 9;
    LlmSanitizeResponseContext response_sanitize_context = 10;
  }
  LlmExecutionCodecContext execution_codec_context = 11;
}
message LlmExecutionCodecContext {
  LlmSanitizeRequestContext request = 1;
  LlmSanitizeResponseContext response = 2;
}
message ToolInvocation {
  string tool_name = 1;
  JsonEnvelope value = 2;
  optional string tool_call_id = 3;
}
message LlmCodecIdentity {
  LlmCodecKind kind = 1;
  optional string id = 2;
}
message LlmSanitizeRequestContext {
  LlmCodecIdentity codec = 1;
  optional string codec_capability_id = 2;
}
message LlmSanitizeResponseContext {
  LlmCodecIdentity codec = 1;
  optional string codec_capability_id = 2;
}
message InvokeResponse {
  oneof result {
    EmptyResult empty = 1;
    JsonResult json = 2;
    GuardrailResult guardrail = 3;
    LlmRequestInterceptResult llm_request = 4;
    WorkerError error = 5;
    ToolExecutionInterceptResult tool_execution = 6;
  }
}

Tool values remain JSON, but the result boundary is structural so Relay can keep an opaque annotation beside the application value without treating either field as a schema-tagged envelope.

ToolInvocation.tool_call_id is set only for tool execution intercepts, and only when the managed tool call recorded a provider-issued identifier. The field is unchanged, so the grpc-v1 identifier and nemo.relay.worker.v1 protobuf package do not change. Beginning with Relay 0.9, worker SDKs surface it through the existing register_tool_execution_intercept callback. A worker plugin that registers this surface must declare a compat.relay range that excludes Relay 0.8, such as >=0.9,<1.0.

message JsonValue {
  bytes json = 1;
}
message ToolExecutionResultResponse {
  ToolExecutionResult value = 1;
  WorkerError error = 2;
}
message ToolExecutionResult {
  JsonValue result = 1;
  JsonValue annotation = 2;
}
message ToolExecutionInterceptOutcome {
  JsonValue result = 1;
  JsonValue annotation = 2;
  JsonValue pending_marks = 3;
}

continuation_id is present only for execution intercepts. scope captures the host context used for continuation and runtime calls. registration_name is the component-local name the worker returned in RegisterResponse; Relay separately owns its qualification in the host registries.

LlmCodecKind has four wire values: unspecified, built-in, runtime, and opaque. A built-in identity carries openai_chat, openai_responses, anthropic_messages, oci_genai, or gemini_generate_content; a runtime identity carries its registered ID. An opaque identity deliberately withholds an ID. The capability ID is optional and invocation-scoped. A worker must treat it as a secret and must not use it after the owner invocation ends.

For an LLM execution invocation, execution_codec_context.request is required. Its identity is present even when no codec resolves, and its capability ID is present only when host codec operations are available. Non-streaming execution also includes response; streaming execution omits it because Relay’s response codecs require a complete provider response. The worker SDK hides both opaque capability IDs behind resolve_codec() proxies.

Host-Runtime Service

The host-runtime request and response fields are complete in the following table:

ScopeContext contains scope_stack_id and parent_scope_id. ScopeType supports agent, function, tool, LLM, retriever, embedder, reranker, guardrail, evaluator, custom, and unknown scopes. The unspecified wire value is invalid for a pushed scope.

Scope timestamps are signed Unix microseconds. Omitting the field preserves the normal host-clock behavior; an explicitly present zero records the Unix epoch. The fields are an additive grpc-v1 extension. Workers that require historical timestamp preservation must declare a Relay compatibility range beginning at 0.10, because an older protobuf host can ignore an unknown field.

LogRequest.level is one of error, warn, info, debug, or trace. Relay prefixes the optional target with its worker-plugin namespace and applies its configured log filter. fields is a lossless JSON object: its keys become structured log attributes, so JSONL sinks preserve nested values rather than flattening them into the message.

EmitMarkRequest.data_schema uses the nemo.relay.DataSchema@1 envelope, severity accepts trace, debug, info, warn, or error, and category is an optional semantic category for the emitted mark. Omitting these fields preserves the original mark behavior. GetRuntimeDiagnostics aggregates repeated codes, retains the latest message, sorts entries by code, and returns at most 32 entries. It is a host-level view, so it does not identify the plugin that recorded a diagnostic.

Authentication and Endpoints

Relay creates a fresh activation ID and high-entropy token, passes the worker and host endpoints through the activation environment, and sends the same values in handshake. Every later worker and host-runtime request includes the activation ID and token. SDKs bind the endpoint locally, prefer Unix domain sockets where supported, constrain TCP fallback to loopback, and reject mismatched credentials. Local authentication prevents accidental cross-activation calls; it does not make untrusted worker code safe.

Relay supplies the following process environment to Rust, Python, and custom-command workers:

JSON Envelopes and Errors

JsonEnvelope contains a schema identifier and UTF-8 JSON bytes. General values use nemo.relay.Json@1; typed request, annotation, outcome, and event schemas identify their own expected shape. The envelope owns its bytes for the message lifetime, so neither side borrows language-runtime objects across RPCs.

The current envelope schemas are as follows:

JsonValue contains exactly one JSON value and carries arbitrary tool results without numeric coercion. ToolExecutionResult requires that value in result and permits an optional annotation; JSON null annotations normalize to absence. A tool execution intercept returns ToolExecutionInterceptOutcome, which adds an optional JSON array of Relay-owned pending_marks. Surface results otherwise use empty, JSON, guardrail, LLM request outcome, or structured error variants. Stream chunks contain one JSON value or one terminal error. WorkerError carries a stable code, human-readable message, and retryable flag; transport errors remain distinct from plugin callback errors.

message JsonEnvelope {
  string schema = 1;
  bytes json = 2;
}
message StreamChunk {
  oneof item {
    JsonEnvelope value = 1;
    WorkerError error = 2;
  }
}
message WorkerError {
  string code = 1;
  string message = 2;
  bool retryable = 3;
}

A clean stream ends when the server stream closes after its last value. A callback failure travels as the terminal error item. Transport cancellation or an unavailable process remains a gRPC status and must not be rewritten into a plugin WorkerError by a custom implementation.

Cancellation and Shutdown Sequence

The host and worker use the following sequence to end active work and release the worker process:

1. Relay stops routing new calls to the component and sends CancelInvocation for in-flight work that cannot drain normally.
2. The worker cooperatively cancels known active invocations and acknowledges only after their callback tasks and stream-owned cleanup have completed. Managed stream close waits for this acknowledgment, including after a terminal stream error. Flush exporters after close to collect final cleanup events. Continuation and codec capabilities become unusable when their owner invocation ends. Unmanaged background work started by a callback remains the plugin’s responsibility.
3. Relay sends Shutdown with the activation ID, token, and reason. The worker stops its service and closes the local endpoint.
4. Relay waits for the managed process within its shutdown policy, terminates it if necessary, removes component registrations, and deletes a managed Python environment only during explicit package removal.

Successful protocol verification covers authentication failures, envelope schema failures, every registration and result variant, repeated unary and incremental stream continuations, cancellation before and during callbacks, codec capability expiry, host runtime ownership errors, health, and orderly plus forced shutdown.

Scope Snapshot Extension

GetCurrentScopeStackSnapshot is additive to grpc-v1. Existing enum numbers and RPCs remain unchanged. ScopeStackEntry.scope_id is a canonical UUID string; scope_type reuses the existing enum. ScopeStackSnapshot.scopes is ordered root to top and includes the implicit root. Snapshot decoders map UNSPECIFIED and future unknown numeric scope types to Unknown.

A missing request scope returns response absence after authentication. A present context with an empty stack ID is invalid. Unknown, foreign, expired, cancelled, and dropped bindings return explicit errors. When supplied, a parent binding must be a live host-issued scope handle owned by the same stack. The getter reads that stack’s visible entries. No ambient thread fallback or invocation-entry cache is used. Host cleanup and reads are ordered so the returned value is coherent and detached.

Older hosts return gRPC UNIMPLEMENTED; SDKs surface an unsupported-operation error distinctly from absence. This operation uses RPC feature detection and requires no change to handshake registration-surface negotiation. Regenerate custom bindings to call it. See the worker example for nearest-Agent correlation and synthetic-root limitations.