grpc-v1 Protocol Reference

grpc-v1 is the stable worker protocol implemented by the Rust and Python SDKs. The current source of truth is crates/worker-proto/proto/nemo/relay/worker/v1/plugin_worker.proto. Generated protobuf types are wire material, not the recommended authoring API.

Relay 0.8 retains the grpc-v1 identifier and nemo.relay.worker.v1 package, but it changes the tool-result boundary to structural protobuf messages. Rebuild workers, regenerate custom bindings, and declare compat.relay beginning at 0.8.0; an earlier worker cannot decode the current ToolNext response or tool-execution outcome.

The protocol consists of the worker-facing service implemented by the plugin process and the host-runtime service implemented by Relay. These are the current service definitions, including cancellation, codecs, and streaming continuations:

1
service PluginWorker {
2
  rpc Handshake(HandshakeRequest) returns (HandshakeResponse);
3
  rpc Health(HealthRequest) returns (HealthResponse);
4
  rpc Validate(ValidateRequest) returns (ValidateResponse);
5
  rpc Register(RegisterRequest) returns (RegisterResponse);
6
  rpc Invoke(InvokeRequest) returns (InvokeResponse);
7
  rpc InvokeStream(InvokeRequest) returns (stream StreamChunk);
8
  rpc CancelInvocation(CancelInvocationRequest) returns (WorkerAck);
9
  rpc Shutdown(ShutdownRequest) returns (WorkerAck);
10
}
11
12
service RelayHostRuntime {
13
  rpc EmitMark(EmitMarkRequest) returns (HostAck);
14
  rpc GetRuntimeDiagnostics(GetRuntimeDiagnosticsRequest)
15
      returns (GetRuntimeDiagnosticsResponse);
16
  rpc PushScope(PushScopeRequest) returns (PushScopeResponse);
17
  rpc PopScope(PopScopeRequest) returns (HostAck);
18
  rpc CreateScopeStack(CreateScopeStackRequest)
19
      returns (CreateScopeStackResponse);
20
  rpc DropScopeStack(DropScopeStackRequest) returns (HostAck);
21
  rpc ToolNext(ToolNextRequest) returns (ToolExecutionResultResponse);
22
  rpc LlmNext(LlmNextRequest) returns (JsonResult);
23
  rpc LlmStreamNext(LlmStreamNextRequest) returns (stream StreamChunk);
24
  rpc DecodeLlmCodecRequest(LlmCodecDecodeRequest) returns (JsonResult);
25
  rpc EncodeLlmCodecRequest(LlmCodecEncodeRequest) returns (JsonResult);
26
  rpc DecodeLlmCodecResponse(LlmCodecDecodeResponse) returns (JsonResult);
27
}

Worker Service

Each lifecycle message carries the following fields. Fields described as envelopes use the schema identifiers in the JSON envelope table later on this page.

Registration Surfaces

The numeric values are part of the wire contract. Unknown value zero never represents a valid registration.

1
enum RegistrationSurface {
2
  REGISTRATION_SURFACE_UNSPECIFIED = 0;
3
  SUBSCRIBER = 1;
4
  EVENT_METADATA_INJECTOR = 2;
5
  TOOL_SANITIZE_REQUEST_GUARDRAIL = 10;
6
  TOOL_SANITIZE_RESPONSE_GUARDRAIL = 11;
7
  TOOL_CONDITIONAL_EXECUTION_GUARDRAIL = 12;
8
  TOOL_REQUEST_INTERCEPT = 13;
9
  TOOL_EXECUTION_INTERCEPT = 14;
10
  LLM_SANITIZE_REQUEST_GUARDRAIL = 20;
11
  LLM_SANITIZE_RESPONSE_GUARDRAIL = 21;
12
  LLM_CONDITIONAL_EXECUTION_GUARDRAIL = 22;
13
  LLM_REQUEST_INTERCEPT = 23;
14
  LLM_EXECUTION_INTERCEPT = 24;
15
  LLM_STREAM_EXECUTION_INTERCEPT = 25;
16
  MARK_SANITIZE_GUARDRAIL = 30;
17
  SCOPE_SANITIZE_START_GUARDRAIL = 31;
18
  SCOPE_SANITIZE_END_GUARDRAIL = 32;
19
}

An invocation names the activation, invocation, registration, surface, optional continuation, captured scope, and token. Its payload is exactly one event, tool invocation, or LLM invocation. LLM sanitizer invocations additionally carry codec identity and an opaque invocation-scoped codec capability.

For EVENT_METADATA_INJECTOR, Relay sends the immutable Event snapshot in InvokeRequest.event. The worker returns an InvokeResponse.json object containing proposed additions to Event.metadata. Relay validates and merges accepted additions before Event sanitizers run. A WorkerError omits that callback’s additions without dropping the Event.

1
message InvokeRequest {
2
  string activation_id = 1;
3
  string invocation_id = 2;
4
  string registration_name = 3;
5
  RegistrationSurface surface = 4;
6
  string continuation_id = 5;
7
  ScopeContext scope = 6;
8
  string auth_token = 7;
9
10
  oneof payload {
11
    JsonEnvelope event = 10;
12
    ToolInvocation tool = 11;
13
    LlmInvocation llm = 12;
14
  }
15
}
16
17
message LlmInvocation {
18
  string model_name = 1;
19
  JsonEnvelope request = 2;
20
  JsonEnvelope annotated_request = 3;
21
  JsonEnvelope response = 4;
22
  reserved 5, 6, 7, 8;
23
  oneof sanitize_context {
24
    LlmSanitizeRequestContext request_sanitize_context = 9;
25
    LlmSanitizeResponseContext response_sanitize_context = 10;
26
  }
27
}
28
29
message ToolInvocation {
30
  string tool_name = 1;
31
  JsonEnvelope value = 2;
32
}
33
34
message LlmCodecIdentity {
35
  LlmCodecKind kind = 1;
36
  optional string id = 2;
37
}
38
39
message LlmSanitizeRequestContext {
40
  LlmCodecIdentity codec = 1;
41
  optional string codec_capability_id = 2;
42
}
43
44
message LlmSanitizeResponseContext {
45
  LlmCodecIdentity codec = 1;
46
  optional string codec_capability_id = 2;
47
}
48
49
message InvokeResponse {
50
  oneof result {
51
    EmptyResult empty = 1;
52
    JsonResult json = 2;
53
    GuardrailResult guardrail = 3;
54
    LlmRequestInterceptResult llm_request = 4;
55
    WorkerError error = 5;
56
    ToolExecutionInterceptResult tool_execution = 6;
57
  }
58
}

Tool values remain JSON, but the result boundary is structural so Relay can keep an opaque annotation beside the application value without treating either field as a schema-tagged envelope.

1
message JsonValue {
2
  bytes json = 1;
3
}
4
5
message ToolExecutionResultResponse {
6
  ToolExecutionResult value = 1;
7
  WorkerError error = 2;
8
}
9
10
message ToolExecutionResult {
11
  JsonValue result = 1;
12
  JsonValue annotation = 2;
13
}
14
15
message ToolExecutionInterceptOutcome {
16
  JsonValue result = 1;
17
  JsonValue annotation = 2;
18
  JsonValue pending_marks = 3;
19
}

continuation_id is present only for execution intercepts. scope captures the host context used for continuation and runtime calls. registration_name is the component-local name the worker returned in RegisterResponse; Relay separately owns its qualification in the host registries.

LlmCodecKind has four wire values: unspecified, built-in, runtime, and opaque. A built-in identity carries openai_chat, openai_responses, anthropic_messages, oci_genai, or gemini_generate_content; a runtime identity carries its registered ID. An opaque identity deliberately withholds an ID. The capability ID is optional and invocation-scoped. A worker must treat it as a secret and must not use it after the owner invocation ends.

Host-Runtime Service

The host-runtime request and response fields are complete in the following table:

ScopeContext contains scope_stack_id and parent_scope_id. ScopeType supports agent, function, tool, LLM, retriever, embedder, reranker, guardrail, evaluator, custom, and unknown scopes. The unspecified wire value is invalid for a pushed scope.

EmitMarkRequest.data_schema uses the nemo.relay.DataSchema@1 envelope, and severity accepts trace, debug, info, warn, or error. Omitting both preserves the original mark behavior. GetRuntimeDiagnostics aggregates repeated codes, retains the latest message, sorts entries by code, and returns at most 32 entries. It is a host-level view, so it does not identify the plugin that recorded a diagnostic.

Authentication and Endpoints

Relay creates a fresh activation ID and high-entropy token, passes the worker and host endpoints through the activation environment, and sends the same values in handshake. Every later worker and host-runtime request includes the activation ID and token. SDKs bind the endpoint locally, prefer Unix domain sockets where supported, constrain TCP fallback to loopback, and reject mismatched credentials. Local authentication prevents accidental cross-activation calls; it does not make untrusted worker code safe.

Relay supplies the following process environment to Rust, Python, and custom-command workers:

JSON Envelopes and Errors

JsonEnvelope contains a schema identifier and UTF-8 JSON bytes. General values use nemo.relay.Json@1; typed request, annotation, outcome, and event schemas identify their own expected shape. The envelope owns its bytes for the message lifetime, so neither side borrows language-runtime objects across RPCs.

The current envelope schemas are as follows:

JsonValue contains exactly one JSON value and carries arbitrary tool results without numeric coercion. ToolExecutionResult requires that value in result and permits an optional annotation; JSON null annotations normalize to absence. A tool execution intercept returns ToolExecutionInterceptOutcome, which adds an optional JSON array of Relay-owned pending_marks. Surface results otherwise use empty, JSON, guardrail, LLM request outcome, or structured error variants. Stream chunks contain one JSON value or one terminal error. WorkerError carries a stable code, human-readable message, and retryable flag; transport errors remain distinct from plugin callback errors.

1
message JsonEnvelope {
2
  string schema = 1;
3
  bytes json = 2;
4
}
5
6
message StreamChunk {
7
  oneof item {
8
    JsonEnvelope value = 1;
9
    WorkerError error = 2;
10
  }
11
}
12
13
message WorkerError {
14
  string code = 1;
15
  string message = 2;
16
  bool retryable = 3;
17
}

A clean stream ends when the server stream closes after its last value. A callback failure travels as the terminal error item. Transport cancellation or an unavailable process remains a gRPC status and must not be rewritten into a plugin WorkerError by a custom implementation.

Cancellation and Shutdown Sequence

The host and worker use the following sequence to end active work and release the worker process:

1. Relay stops routing new calls to the component and sends CancelInvocation for in-flight work that cannot drain normally.
2. The worker acknowledges known active invocations and cooperatively cancels their async tasks. Continuation and codec capabilities become unusable when their owner invocation ends.
3. Relay sends Shutdown with the activation ID, token, and reason. The worker stops its service and closes the local endpoint.
4. Relay waits for the managed process within its shutdown policy, terminates it if necessary, removes component registrations, and deletes a managed Python environment only during explicit package removal.

Successful protocol verification covers authentication failures, envelope schema failures, every registration and result variant, repeated unary and incremental stream continuations, cancellation before and during callbacks, codec capability expiry, host runtime ownership errors, health, and orderly plus forced shutdown.