Supervisor Middleware

Supervisor middleware adds ordered processing stages to allowed HTTP and WebSocket egress. Middleware runs after network and L7 policy admit traffic and before OpenShell injects provider credentials. A stage can allow or deny an HTTP request or client WebSocket text message, replace its payload, add approved HTTP headers, and report audit-safe findings.

Middleware selection is independent of the network policy rule that admitted the request. OpenShell matches middleware by destination host, so the same middleware applies consistently across broad, specific, user-authored, and provider-derived network policies.

Request Flow

For each inspected HTTP request, the supervisor:

1. Evaluates network and L7 policy.
2. Selects middleware whose host selectors match the admitted destination.
3. Buffers the request body using the largest body limit in the selected chain.
4. Runs matching middleware by ascending order. Policy validation rejects duplicate order values.
5. Re-checks body-aware protocol policy (GraphQL, JSON-RPC, MCP) after each stage that replaces the body. Every middleware receives a payload the policy admits, and a transformation cannot smuggle a denied or unparseable operation to a later stage or the upstream.
6. Applies allowed transformations, injects provider credentials, and forwards the request.

For an RFC 6455 upgrade over ws:// or wss://, the supervisor first finds every host-matched attachment, then selects only implementations that advertise WEBSOCKET_MESSAGE/PRE_CREDENTIALS. It opens one ordered, phase-specific EvaluateWebSocketSession stream per selected stage. OpenShell sends WebSocketSessionEvent values, while the service returns WebSocketSessionEventResult values only for preflight and message events; session start and end are notifications. Future upstream-to-client inspection uses the same RPC with PRE_RETURN; an implementation that advertises both phases receives two independent streams for the WebSocket session. An attachment without the selected binding can still inspect the HTTP upgrade request when it advertises the HTTP binding, but it is not a failed WebSocket stage. OpenShell allows post-upgrade traffic and emits an informational binding_not_selected coverage event for that attachment.

1. A preflight before the upgrade is sent upstream. The stage chooses INSPECT, voluntary SKIP, or authoritative DENY and may return a bounded diagnostic reason, stable reason code, findings, and metadata. OpenShell runs selected preflights concurrently; any DENY rejects the upgrade regardless of on_error.
2. A session-start event after the upstream accepts the upgrade, including the negotiated subprotocol.
3. Complete client-to-upstream text messages in sequence order. OpenShell reassembles fragmented messages and decompresses negotiated permessage-deflate messages before evaluation.
4. A best-effort session-end event when the stage stream remains writable. OpenShell attempts at most one terminal event for each opened stream, including streams opened during a preflight that rejects the upgrade before session start.

The protobuf represents each logical message with a text or binary payload variant. Text uses the protobuf string type, so invalid UTF-8 cannot enter the middleware contract. Results use an optional matching replacement variant: absence preserves the input, while presence represents a replacement even when its content is empty. OpenShell rejects attempts to change the message type. Allowed replacements are re-framed, re-compressed when required, and forwarded. Binary messages, control frames, and upstream-to-client traffic remain uninspected. Binary messages pass through under both on_error modes. For each active selected stage, OpenShell emits an informational unsupported_message_type coverage event and advances the session-global sequence; the next text message can therefore reach the stage with a valid sequence gap.

The network supervisor reserves process-wide assembly capacity before buffering every parsed WebSocket text message, even when no middleware is selected. At most 32 assemblies run while 64 additional callers wait without buffering payload bytes. When both bounds are full, OpenShell closes the WebSocket with code 1013 before reading the new message payload. A text message may contain at most 4,096 fragments, must make input progress within 30 seconds, and must finish assembly within 2 minutes. Forwarding the completed text frame must finish within another 2 minutes. The assembly budget lasts for the supervisor process lifetime, so policy reloads do not reset its capacity.

Active middleware sessions additionally reserve shared middleware capacity before buffering WebSocket text, and HTTP middleware reserves the same capacity before buffering request bodies; at most 32 evaluations run and 64 additional unbuffered callers wait for capacity. When both middleware bounds are full, OpenShell sheds an HTTP request with 503 Service Unavailable before reading its body. Persistent middleware streams use a separate process-wide budget of 32 sessions. WebSocket session admission does not wait: if the budget is full, OpenShell applies each selected config’s on_error behavior before opening a stream.

Because each transformed body is re-checked before the next stage runs, a middleware hook always receives a request that satisfies the sandbox policy. A stage whose output the policy rejects stops the chain; under enforcement: audit the rejection is logged and the request proceeds.

If post-transformation policy evaluation itself fails, OpenShell denies the request and emits a high-severity detection finding. This failure is separate from middleware on_error because the middleware completed successfully; the sandbox policy could not validate its output.

Middleware receives the request before credential injection. Operator-run services cannot inspect OpenShell-managed credentials. Middleware-visible request headers are delivered in wire order and repeated header names are preserved as separate entries. OpenShell filters credential, routing, framing, and hop-by-hop headers before invoking middleware. It rejects malformed request headers and unsupported transfer-coding sequences before middleware or policy dispatch. Headers named by a request’s Connection field are omitted from middleware input and removed before forwarding, except for the validated WebSocket upgrade pair.

The request context identifies the originating sandbox to operator-run services. It carries the sandbox ID (sandbox_id), the sandbox name (sandbox_name), and the workspace (workspace), letting audit and approval interfaces show a human-readable name and its workspace instead of an opaque ID. sandbox_name and workspace are for display and logging only: names are workspace-scoped and may be reused for different sandbox instances, so services must use sandbox_id for authorization, persistence, durable correlation, and identity. sandbox_id is always present on middleware requests. sandbox_name and workspace are best-effort: a supervisor that cannot resolve a value, or an older supervisor that predates a field, sends an empty string. Services should fall back to the sandbox ID when the name or workspace is empty.

Choose a Middleware Type

openshell/regex is an example built-in middleware. It replaces only simple, self-contained token patterns in UTF-8 HTTP bodies and client WebSocket text messages; the initial pattern recognizes sk- tokens. It does not infer values from keyword assignments such as JSON password fields. This best-effort text transformation is not parser-aware and does not guarantee that it will detect or fully remove sensitive values. Its config accepts one field, mode: redact, which is also the default when the field is omitted. Unknown config fields and non-string values are rejected at policy validation. Custom expressions are not configurable yet.

Operator-run services expose bindings for supported operation and phase pairs. A binding is identified by its operation and phase. V1 supports HttpRequest/pre_credentials and WebSocketMessage/pre_credentials; a service may expose either or both. Policies attach the complete middleware by its operator-owned gateway registration name.

Register a Middleware Service

Start an operator-run service before starting the gateway, then add a registration to the local gateway TOML:

1
[[openshell.supervisor.middleware]]
2
name = "local-content-guard"
3
grpc_endpoint = "https://content-guard.example:50051"
4
tls_ca_cert_path = "/etc/openshell/content-guard-ca.pem"
5
audience = "urn:example:content-guard"
6
max_payload_bytes = 262144
7
timeout = "500ms"

Each binding returned by Describe may advertise a shorter timeout using the same syntax and bounds. The operator-configured service timeout is a ceiling: OpenShell uses the smaller of the binding and service values. An omitted binding timeout inherits the service setting, and an omitted service setting uses the 500 ms platform default. OpenShell rejects an invalid timeout before accepting the manifest. The operator-configured service timeout applies to Describe and ValidateConfig. The effective binding timeout applies only to EvaluateHttpRequest, WebSocket preflight, and each WebSocket message. WebSocket streams have no connection-wide deadline.

The gateway connects to every registered service and verifies its capabilities before accepting traffic. Gateway startup fails when a service is unavailable, reports an invalid capability, or exposes more than one binding for the same operation and phase. The manifest name is diagnostic metadata and does not need to match the operator registration name. Operator-run registration names cannot claim the reserved openshell/ namespace.

Registration is static. Restart the gateway after adding, removing, or changing a service. See Gateway Configuration for the complete gateway TOML context.

Authenticate OpenShell Callers

When gateway JWT signing is configured, OpenShell attaches a short-lived EdDSA bearer token to every remote middleware RPC. Gateway calls use caller_kind: gateway; sandbox supervisor calls use caller_kind: supervisor and include the sandbox ID. Supervisors request credentials by registration name through RefreshSandboxToken. The gateway derives the audience from operator-owned configuration and authorizes each name against the sandbox’s effective policy.

Return your expected audience in the expected_audience field of your Describe manifest. After authenticated Describe succeeds, OpenShell compares the advertised value with its operator-configured audience and refuses to start when they differ. This is a post-authentication consistency assertion, not audience discovery: a strict verifier may reject an incorrect audience before returning the manifest, in which case startup reports an authentication failure. Leave the field empty to skip the consistency check.

Provision the trusted gateway URL, expected gateway ID, and public key or JWKS through the deployment. This operator-provisioned key material is the authoritative cold-start trust anchor. The expected issuer is exactly openshell-gateway:<gateway_id>; fetching JWKS does not establish that identity by itself. After initial trust is established, GET /.well-known/openid-configuration and its jwks_uri provide steady-state key refresh and operational convenience. The document is OIDC-shaped rather than OIDC-compliant: issuer is the gateway identity, not the URL serving the document, so compare iss against the configured value and fetch updates only over authenticated TLS at the trusted gateway URL.

Cache keys by kid. Validate, at minimum:

• typ is exactly openshell-ext+jwt. Extension tokens and sandbox-to-gateway bootstrap tokens share a signing key and differ only in audience; this header is a second, independent discriminator.
• alg is pinned to EdDSA. Never select the algorithm from the token.
• Signature, expected issuer, exact audience, and positive expiry.
• caller_kind, and the sandbox identity when your service scopes behavior per sandbox.

A sandbox-to-gateway JWT is not an extension credential even though both token types use the same signing key.

Each token carries a unique jti that identifies that token instance for correlation and future explicit revocation. OpenShell reuses a token across calls until rotation and does not track jti, so rejecting a repeated jti would reject legitimate requests. Per-request replay resistance requires a request nonce or signature, channel binding, or another proof-of-possession mechanism.

Run Without Extension Authentication

Set allow_insecure_transport = true on a registration to keep a plaintext http:// endpoint working. OpenShell then attaches no credential to that service, supervisors do not request one, and the gateway refuses to mint one if asked. The gateway logs a warning naming the registration at every startup.

The service cannot distinguish OpenShell from any other client that can reach it. Use this only where the network already provides that guarantee, and prefer https:// everywhere else.

Apply Middleware with Policy

Add middleware configs to the top-level network_middlewares map. Each key is the policy-local config name:

1
network_middlewares:
2
  regex-redactor:
3
    name: Redact API tokens
4
    middleware: openshell/regex
5
    order: 10
6
    config:
7
      mode: redact
8
    on_error: fail_closed
9
    endpoints:
10
      include: ["*.example.com"]
11
      exclude: ["trusted.example.com"]

Each config has a stable policy-local identity from its map key, an optional human-readable name that defaults to that key, a built-in or operator-owned registration name in middleware, an integer order, implementation-owned config, failure behavior, and host selectors. The optional name does not replace the map key for attachment or future keyed updates. A policy accepts at most 10 middleware configs.

include selects destination hosts. exclude takes precedence and removes hosts from that selection. Each config accepts at most 32 combined include and exclude patterns. Matching is case-insensitive and uses the same exact-host and DNS glob behavior as network policy endpoints: * matches exactly one DNS label, ** matches one or more labels, and intra-label patterns like *-api.example.com work. Brace alternates such as {prod,staging} are rejected at validation; list each host pattern separately.

Matching configs run once each by ascending order; lower values run first. Order values must be unique across the complete policy, even when endpoint selectors do not overlap. The default order is 0, so policies with multiple configs normally set explicit values. Different map keys may attach the same middleware and run as separate stages. Map keys are structurally unique. Runtime selection defensively rejects chains with more than 10 stages.

See Policy Schema for the complete field reference.

Configure Failure Behavior

on_error controls what happens after an operation binding is selected and middleware is unavailable, rejects its configuration, returns an invalid result, or exceeds the selected binding’s payload limit. It does not turn an unadvertised operation or an unsupported WebSocket message class into a middleware failure.

Use fail_open only when bypassing the middleware preserves the intended security policy. OpenShell emits a detection finding when a failed stage is bypassed and a separate state-change finding when a WebSocket stage is disabled for the session.

Capability coverage is separate from failure handling. A host-matched HTTP-only attachment does not join the WebSocket chain, regardless of on_error. Binary messages are outside the V1 text-message binding and pass through even when a selected stage is fail_closed. OpenShell records both states as informational coverage events so operators do not mistake pass-through traffic for inspected traffic. If a deployment requires all WebSocket message classes to be inspected, V1 cannot express that requirement.

An explicit deny decision always stops the chain and denies the request or WebSocket upgrade, regardless of on_error. A WebSocket preflight DENY is a successful policy decision, not a middleware failure; OpenShell rejects the upgrade before upstream contact and ends each still-writable stream opened by a successful preflight decision with MIDDLEWARE_DENIAL. The HTTP response uses error: middleware_denied, identifies the policy-local middleware config, and omits policy-advisor remediation because the network and L7 allow rules already matched. OpenShell never copies the free-form middleware reason into the response or security logs. HTTP results, WebSocket preflight decisions, and WebSocket message results can instead return an optional stable reason_code: 1–64 bytes, starting with a lowercase ASCII letter and containing only lowercase ASCII letters, digits, and underscores. Invalid codes make the result a middleware failure governed by on_error. Preflight findings and metadata use the same bounds and audit-safe handling as message results.

1
{
2
  "error": "middleware_denied",
3
  "detail": "Request rejected by configured middleware",
4
  "policy": "api-policy",
5
  "middleware": "prototype-content-guard",
6
  "reason_code": "content_match"
7
}

A failed fail_closed stage uses error: middleware_failed and a platform-owned detail. It also omits rule_missing, next_steps, and agent_guidance: the failure did not result from a missing network or L7 policy rule, and changing policy cannot repair it. Runtime diagnostic text is available only through sanitized operator telemetry.

Middleware decisions are enforced regardless of the endpoint’s enforcement mode. enforcement: audit applies to an endpoint’s network and L7 policy rules and does not bypass middleware: a middleware deny, or a failed fail_closed stage, blocks the request even on an audit endpoint. A middleware service that needs to observe traffic without blocking should return an allow decision with findings, which OpenShell emits as detection findings.

Set Payload Limits

Every middleware binding declares the largest logical payload or replacement it supports through max_payload_bytes. For HTTP_REQUEST, that payload is one request body. For WEBSOCKET_MESSAGE, it is one complete message rather than the whole session.

• Built-in middleware uses its OpenShell-defined limit.
• Each operator-run registration sets one max_payload_bytes ceiling no higher than any binding’s advertised max_payload_bytes capability.
• A selected chain buffers using its largest stage limit, so every stage that can process the body receives it.
• The same per-stage limit applies to request bodies and replacement bodies.

The gateway rejects a registration whose operator limit exceeds the service capability or the 4 MiB platform maximum instead of silently clamping it. OpenShell also bounds the non-payload protobuf components: 64 KiB for service config, 4 KiB for request context, 32 KiB for the target, and 128 request header lines totaling at most 64 KiB encoded. Results allow a 4 KiB discarded free-form reason, a 64-byte validated reason code, 64 header mutations totaling at most 64 KiB encoded, 32 findings of at most 4 KiB encoded each, and 64 metadata entries totaling at most 32 KiB. Middleware gRPC servers should configure request and response message limits to at least 4 MiB plus 293 KiB so every platform-valid envelope fits.

At request time, exceeding a selected stage’s limit is a middleware failure for that stage alone and follows that config’s on_error behavior; other stages in the chain still run against their own limits. OpenShell can apply fail_open to an oversized Content-Length before consuming body bytes. A chunked body can cross the limit only after bytes have been consumed, so OpenShell denies that request because it cannot safely resume the original stream.

For a WebSocket binding, max_payload_bytes covers complete client text messages and replacements. Exceeding a selected stage’s effective text-message limit follows that stage’s on_error. The 4 MiB parsed-text platform cap and other protocol-safety limits are independent of middleware failure policy. Binary messages are not delivered to middleware, so the operator ceiling does not become a binary relay limit; individual raw binary frames retain the 16 MiB relay-safety bound. Oversized parsed text closes the connection with code 1009; invalid UTF-8 uses 1007; protocol errors use 1002; middleware or policy denials use 1008; and policy reload uses 1012.

Mutate Request Headers

A middleware result can return ordered header mutations before OpenShell injects credentials. A write mutation adds a value when the case-insensitive header name is absent and selects one behavior when it is already present:

• append adds another field value.
• overwrite removes every existing value before adding the new value.
• skip leaves existing values unchanged.

A remove mutation removes every value for a case-insensitive header name. OpenShell applies each successful stage’s mutations before invoking the next middleware, so later stages observe the accumulated header state.

Header writes must use the x-openshell-middleware- prefix. Removes may target other middleware-visible request headers. Protected credential, routing, framing, and hop-by-hop headers are always rejected. Header values must not contain control characters.

OpenShell validates and applies each stage’s mutations atomically. An invalid operation discards every mutation from that stage and follows its on_error behavior. Built-in failures can name the offending header. Operator-run failures use a platform-owned error code so request-derived header text cannot reach logs or denied responses.

Operate Middleware Services

Plan startup and updates around these boundaries:

• Start registered services before the gateway. The gateway validates every registration during startup.
• Keep service endpoints reachable from both the gateway and sandbox supervisors. The supervisors call operator-run services directly on the request path.
• Restart the gateway after changing registrations.
• Keep required services available before creating or updating policies. The gateway validates implementation-owned config before persisting a policy.
• Treat fail_open as an explicit availability-over-enforcement decision.

When the effective sandbox configuration changes, a running supervisor validates the new service registry before installing it. If the reload fails, the supervisor keeps its last-known-good registry and emits a configuration failure event.

Observe Middleware

Middleware activity is emitted through OpenShell’s OCSF logging:

• Each invocation records its policy-local config name, attached middleware name, decision, transformation state, and failure state.
• A denied invocation records a platform-owned reason derived from the policy-local config name and optional validated reason code. OpenShell does not record service-provided free-form reason text.
• A bypass under fail_open emits a detection finding.
• A required stage that fails closed emits a high-severity detection finding.
• A host-matched attachment without a WebSocket binding emits an informational binding_not_selected coverage event.
• A binary message encountered by an active WebSocket stage emits an informational unsupported_message_type coverage event with message type, sequence, and byte count. It is not reported as an invocation or failure.
• Built-in findings include their type, label, and aggregate count. Operator-run findings use the operator-owned registration name and a platform label plus the aggregate count; OpenShell does not log service-provided finding text or diagnostic metadata. A stage can return at most 32 findings. Exceeding the per-stage cap is an invalid response handled through on_error. A maximum 10-stage chain retains and emits up to 320 findings without silently dropping findings from later stages.
• Registry reload success and failure are emitted as configuration state changes.

See Logging for log access and OCSF JSON Export for structured export.

Current Limitations

• Middleware applies only through operation bindings advertised by each implementation. For protocols that have no supported middleware operation at all, such as HTTP/2 prior knowledge or non-HTTP TCP, the existing uninspectable-traffic gate denies a host match containing fail_closed and relays an all-fail_open match with a detection finding.
• The typed operation and phase pairs are HTTP_REQUEST/PRE_CREDENTIALS and WEBSOCKET_MESSAGE/PRE_CREDENTIALS.
• A host match does not imply every advertised operation: an HTTP-only attachment can inspect the upgrade GET, then post-upgrade traffic passes with binding_not_selected coverage.
• The V1 WebSocket binding inspects complete client text messages only. Binary messages pass with unsupported_message_type coverage for active stages; control frames and upstream-to-client messages remain outside the middleware operation.
• Selection uses destination host include and exclude patterns.
• A fail-closed middleware cannot cover tls: skip endpoints because OpenShell cannot inspect that traffic. An all-fail_open match may cover the endpoint; OpenShell bypasses the middleware and emits a detection finding.
• Operator-run services use TLS https:// when gateway JWT signing is enabled, unless the registration sets allow_insecure_transport. Certificates must chain to the configured custom CA or platform roots, and the endpoint hostname must match.
• Extension tokens and sandbox-to-gateway tokens are signed by the same key. They are separated by audience and by typ, but the extension credential path cannot yet be rotated or revoked independently of sandbox admission.
• OpenShell does not track or revoke jti; bearer tokens can be replayed until expiry. Per-request replay resistance requires proof of possession or request binding.
• mTLS client authentication, health checks, runtime registration, and overlapping signing-key rotation are not available.