Migration Guides

Use this page to upgrade from NeMo Relay 0.10.0 to 0.11.0.

This guide covers the 0.11.0 release. It includes fixes from the 0.10 patch releases. Configuration changes from earlier patch upgrades still apply.

Start with the package updates. Then review the sections for the features you use.

Upgrade to NeMo Relay 0.11.0

Update Packages and Plugin SDKs

Update the Relay CLI, application bindings, and plugin SDKs to the same release. Rebuild Rust applications and native or worker plugins with the updated dependencies. Update the OpenClaw plugin if you use it. Its Node.js binding dependency must match the plugin version.

Node.js applications and package workflows now require Node.js 24.13.1 or later. Update the runtime before installing the new binding or OpenClaw plugin.

For package names and installation commands, refer to Installation. For a managed daemon deployment, follow Upgrade and Roll Back. Keep the daemon’s identity state and existing managed bundle during the upgrade.

Integrations and Coding Agents

Review LangChain Tool-Call Shapes in Middleware

In 0.11.0, managed LangChain and LangGraph tool results use the nemo.relay.tool_result@1 JSON wrapper. It holds display content, failure status, and the result values with their types.

Review custom Relay middleware that reads or changes LangChain and LangGraph tool calls. Check both the arguments it receives and the results it returns:

• Tool arguments remain JSON. Check that middleware edits match the tool’s input schema.
• Tool results use a wrapper with schema, content, is_error, and result fields. Set schema to nemo.relay.tool_result@1. Edit content to change the displayed result. Set is_error to true for failure or false for success.
• Each value in result includes its type. Update middleware and subscribers that read or edit this field. Keep parts with no JSON form unchanged.
• If an execution intercept returns a result without running the tool, return the full wrapper. The framework’s codec must be able to decode its result.

Later calls can restore built-in values and supported framework types. Custom Python objects stay with the call that returned them. Their data is visible in the wrapper when it has a JSON form. The response cache runs those tools again instead of replaying the objects. For the value format and replay rules, refer to Normalized Tool Results and LangChain.

Check ADK LiteLLM Settings

The Google Agent Development Kit (ADK) integration supports LiteLlm with litellm 1.106.2 or later. Relay records the provider LiteLLM selects. If an intercept selects a model from another provider, Relay fails the call.

Relay rejects LiteLLM settings that can switch providers during a call. This includes provider fallbacks and model_list. Use ADK FallbackModel to try another provider if a call fails. Also review your prompt management and callback settings. The full list of rejected settings is in Supported Models and Providers.

Review Deep Agents Middleware Order

For Deep Agents, set position="outer" in add_nemo_relay_integration to record tool results after your middleware changes them. The default remains "inner". It records results before your middleware changes them. Refer to Middleware Ordering.

Review CLI Turn Identity

CLI turn spans now inherit the agent name from their owning Agent scope. Paired tool hooks use that name unless they supply an explicit name. Review dashboards that group spans by gen_ai.agent.name or span name. These values can now use a configured agent name instead of a default such as codex. Generic gateway turns omit the agent name unless you supply one. Refer to Turn Identity.

Refresh Hooks and Restore Credentials

Restart Relay gateways and daemons after upgrading. Local hook delivery now uses Transport Layer Security (TLS). Relay checks the server’s identity before sending credentials or payloads.

If a loopback daemon runs under another account, publish its hook certificate for local users. This applies to daemon addresses that use http://. Refer to Publish the Hook Certificate.

Refresh personal Relay-managed Codex and Claude Code integrations before starting new sessions. Run the following command:

nemo-relay integrations refresh

To refresh one Codex or Claude Code integration, use nemo-relay install <agent> --force. For a personal Relay-managed Pi extension, run nemo-relay install pi --force. Replace manually installed Pi extensions with the updated source. For managed daemon bundles, follow Upgrade and Roll Back instead. Do not regenerate those bundles with personal install commands.

Generated tool checks use PreToolUse. Relay no longer generates a separate PermissionRequest hook. Without an explicit policy flag, tool checks and unreadable or invalid hook payloads fail closed. Other named events fail open. A fail-closed hook blocks the operation if Relay cannot check it. Review custom wrappers that omit a policy flag.

Fail-closed delivery failures now return exit code 2. This includes missing credentials, connection errors, timeouts, and HTTP errors. Fail-open delivery failures return 0. Guardrail rejections return 2 under either policy. Hook requests now have a 30-second timeout. Leave room for delivery when setting guardrail evaluation timeouts. Refer to Hook Forwarding.

Hook sessions now use a valid TRACEPARENT from the launching process as their OpenTelemetry parent. TRACESTATE travels with it. The first hook that creates a session sets its parent; later hooks do not change it. Review launch scripts that already set these variables. Unset them if each session should start a new trace. Refer to Continue the Launcher Trace.

Make sure hooks can read NEMO_RELAY_CLIENT_TOKEN or the per-user token file. If the file is missing, run the following command as the user who runs the harness:

nemo-relay daemon token ensure

Then start a new harness session so the Model Context Protocol (MCP) server registers the credential. Model requests need the route credential in the harness’s provider headers. Creating the token file alone does not add that header. Managed Pi also requires NEMO_RELAY_CLIENT_TOKEN in its environment. Refer to Credential Resolution.

Replace Managed Claude Hook Bundles

NeMo Relay 0.11 adds StopFailure to the existing managed v1 Claude bundle. This is a breaking change for previously generated managed bundles containing Claude Code: doctor, managed refresh, and output reuse reject their old canonical bytes. Bundles containing only Codex or Pi are unaffected.

Before upgrading a managed Claude deployment, generate a replacement bundle with the same daemon address, dispatcher, platform, and agents, using a new output directory. Deploy it, provision the new SHA-256 separately, and update the Claude plugin and settings paths to the replacement directory. Refresh remains validation-only and does not rewrite the old bundle. Refer to managed bundle configuration for the generation and validation commands.

Review Recovered Tool Completions

Relay can now close a Codex MCP tool span when a later model request contains the matching result. Model traffic must pass through Relay for this recovery. The outcome is unknown, and the duration can exceed the actual run time. This recovery does not apply to shell tools.

Review dashboards that treat all closed tool spans as successful. To export the recovery fields as OpenTelemetry span attributes, add "tool_completion_" to the trace endpoint’s promote_metadata_prefixes. Refer to Codex Captured Events.

Diagnostics and Observability

Enable and Protect Diagnostics

Runtime diagnostics is optional and disabled by default. Existing logging and exporters do not enable it. For gateways and daemons, configure [diagnostics] in config.toml. Embedded applications use plugins.toml or the diagnostics profile passed to plugin.initialize.

The first embedded initialization selects the process settings, even when collection is disabled. Later initialization with different settings returns diagnostics: configuration_conflict. Restart the process to change them. For an existing gateway, update its owner’s configuration and restart it. Capture any needed evidence first because shutdown discards the stored history. Refer to Embedded Diagnostics and CLI and Daemon Diagnostics.

Grant diagnostic read access only to callers allowed to inspect the whole context. Detail can contain private data that has not been sanitized. Query filters do not restrict a reader’s access. Native plugin reads need an explicit native_diagnostics_read_context policy grant. Worker plugin recording does not grant access to host snapshots. Refer to Plugin Diagnostics.

Diagnostic memory limits cover diagnostic storage, not total process memory. Daemon worker caches can contain only part of a snapshot or a stale page. Check the returned coverage and freshness fields. Refer to Diagnostic Memory Inspection and Diagnostics API.

Review Exported Telemetry

Review stored events, dashboards, and custom exporters for the changes below. These include Agent Trajectory Interchange Format (ATIF) output and personally identifiable information (PII) filtering.

Review cost estimates that include cached input. Relay now bills uncached input, cache reads, and cache writes separately. It counts all input tokens when choosing a pricing tier. If token counts conflict, Relay does not estimate a cost. Refer to Model Pricing.

Remote ATIF uploads and local recovery files now use compact JSON, with fewer spaces and line breaks. Use a JSON parser to read these files. The new include_observed_events and include_llm_requests options both default to true. Set both to false to remove both copies of full requests. Refer to Reduce Output Size.

Daemon workers now pass request codecs to telemetry sanitizers. For manual Rust calls, supply a codec with LlmCallParams::builder().codec(...). PII rules that target normalized fields omit the telemetry payload if no suitable codec is available. The provider request stays unchanged. Refer to Provider Codecs.

For private collectors, you can now set Transport Layer Security (TLS) certificate and client key paths on each OpenTelemetry Protocol (OTLP) endpoint. Prefer HTTPS. Use insecure_http_hosts only when a destination needs an explicit exception to the plaintext transport checks. Refer to TLS and Mutual TLS and Insecure HTTP Hosts.

Automatic OTLP trace, log, and metric exporters now require HTTPS for remote destinations, even without headers. Check your OTEL_* endpoint variables. To allow HTTP for a trusted host, list it in NEMO_RELAY_OTEL_INSECURE_HTTP_HOSTS. This setting applies only to automatic exporters. Prefer HTTPS because HTTP sends data without encryption. Refer to Automatic OpenTelemetry Configuration.

Make TLS files available when Relay checks plugins and when exporters start. Relay checks that it can read the PEM files and that client certificates match their keys. A fatal TLS error stops an HTTP trace exporter. Fix the TLS problem, then recreate the exporter. Traces created while it is stopped are lost. Calls to force_flush() and plugin cleanup can now report this delivery error. Refer to Trace Batch Processor Configuration.

Run nemo-relay doctor after checking exporter settings. It now checks file-sink paths, disabled components and destinations, references to credentials, and PII settings. Checks for files and credential references also run with --offline. A passing HTTP check only means the collector answered. It does not prove that the collector accepts your credentials or trace data. Doctor does not test whether redaction removed sensitive data. It also does not read header_file values. Refer to Check Configuration and Exporters.

Providers and Request Handling

Check Provider URLs

Use HTTPS for remote provider base URLs. This includes private Kubernetes Service addresses. By default, plain HTTP is allowed only for loopback hosts, such as localhost, 127.0.0.1, and ::1. Remove user information, query strings, and fragments from base URLs. Relay checks configuration files, environment variables, and command-line overrides at startup.

For an unauthenticated OpenAI- or Anthropic-compatible inference gateway, you can allow remote HTTP with [upstream] insecure_http_model_hosts. List each trusted hostname or IP address explicitly. These requests remove credential and custom headers, ignore environment API key fallbacks, and do not run API key helpers. Startup fails if that provider has a configured API key, authentication header, or helper. Prefer HTTPS because HTTP sends model content without encryption. Refer to Credential-Free HTTP Model Upstreams.

Relay no longer follows upstream redirects. It also removes the Location header from redirect responses. Set the final provider URL directly. Refer to Provider Upstreams.

Review Managed Provider Credentials

Direct gateway clients must authenticate to Relay before using credentials from Relay’s configuration or environment. Use nemo-relay run to set up client access, or send provider credentials from the client.

Anonymous daemon requests can use daemon-held credentials only in explicit --pass-through mode on a loopback listener. A daemon bound to 0.0.0.0 requires a bound route for this access. This also applies to requests sent to 127.0.0.1. Refer to Requests Without a Bound Credential.

Managed provider requests and model discovery now use the same credential rules. A configured API key helper replaces the caller’s provider credential. Without a helper, Relay keeps caller-provided API keys. It can replace native ChatGPT credentials with an administrator’s OpenAI credential on managed routes. If Relay has no replacement, OpenAI requests keep the caller’s ChatGPT credential. Review deployments that use a helper or an administrator credential to choose the provider account. For the credential selection order, refer to Provider Upstreams.

GET /models and GET /v1/models use Anthropic when the request includes anthropic-version or anthropic-beta. Otherwise, they use OpenAI. The API key’s format does not select a provider. Model lists do not create model events.

Compaction, voice, and other unmanaged routes do not receive Relay-held provider credentials. Supply any required credentials from the client. With a custom openai_base_url, Relay removes recognized ChatGPT credentials from these requests. ChatGPT voice call creation at POST /backend-api/codex/realtime/calls keeps the caller’s ChatGPT credential.

For Codex, a custom openai_base_url now also applies to model requests that keep ChatGPT credentials. Without a custom base or replacement credential, Relay uses the native ChatGPT Codex backend. To change that backend, set [upstream] chatgpt_codex_base_url to an endpoint trusted with the caller’s ChatGPT credentials. Refer to Codex Responses Compatibility.

Managed Claude model routing requires an Anthropic API key or an administrator’s Anthropic-compatible gateway credential. Claude.ai OAuth is not supported with ANTHROPIC_BASE_URL. If your deployment uses only OAuth, keep native routing instead of setting the managed base URL. Refer to Daemon Configuration.

Review Pi Provider Endpoints

Named upstreams for Pi now require public or loopback addresses. Private, link-local, and other special-use addresses are refused. This check also applies to addresses found through DNS. Remote endpoints require HTTPS.

For a private provider endpoint, configure the gateway’s OpenAI or Anthropic base URL instead of using a named upstream. Managed deployments set these URLs in system configuration. Refer to Model Redirection.

Review Request Intercepts

Relay now preserves provider fields across multiple request edits. It keeps fields on unchanged entries and on tool calls or results with a clear call-ID match. New entries do not inherit fields from old entries. Check intercepts that edit several messages or array entries. Refer to Editing Arrays and Mappings.

LangChain model intercepts can now change tool descriptions, input schemas, and provider-specific tool definitions. Keep tool count, order, and names unchanged. Check that the model adapter accepts the edited definitions.

Verify Worker Middleware

Daemon workers now run all six large language model (LLM) middleware types. This applies to OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages, including streaming calls. Review plugins that can now change or block requests and responses.

When request intercepts, execution intercepts, or conditional execution guardrails are active, successful calls return HTTP 200. They do not keep the provider’s original success headers or trailers. Stream formatting can also change. Sanitizers alone leave the provider response unchanged. Refer to LLM Middleware.

After the upgrade, verify worker status and an observable plugin result. /healthz and MCP readiness alone do not prove that plugins are active. Use --require-worker when requests must wait for worker processing. Refer to Verify Worker-Backed Operation.

Review HTTP Error Handling

Remote NeMo Guardrails failures now preserve the received HTTP status through FlowError::Upstream. Update Rust error matches that expected FlowError::Internal.

Plugin error messages with the exact form HTTP <status>: <message> now carry an HTTP status. The status must be from 400 through 599. Review existing callback errors that use this form. Use the SDK’s http_error_message helper to create one. After streaming headers are sent, an error cannot change the HTTP response status. Refer to Build Native Plugins and Worker Middleware and Continuations.

Configuration and Deployment

Review Configuration Loaded by Workers

Local and remote-client daemon workers now load the user’s config.toml, plugins.toml, and dynamic plugins alongside the system configuration. System settings take priority. Observability destination lists can include entries from both layers.

Review settings and plugins that workers previously ignored. Then restart the affected harness sessions to load the updated configuration. Managed workers still ignore provider and gateway environment overrides, except for supported authentication variables. Refer to Configuration and Managed Clients.

Keep Policy Checks Before Caches

Large language model (LLM) and tool response caches now use priority 150 by default. NeMo Guardrails uses 100. Lower values run first. Review explicit priorities so policy checks run before cache lookups, including on cache hits. The LLM cache does not run when remote NeMo Guardrails handles a call. Refer to Response Cache.

Review Operational Logging

Logging overrides now change only the fields they set. For example, NEMO_RELAY_LOG=info keeps file sinks, rotation settings, and the flush interval from TOML configuration. Review scripts that relied on an override to replace the full logging configuration. Refer to Operational Logging.

File sinks remain required by default. Set failure_policy = "optional" on an individual sink to let Relay start even if it cannot open that file. If Relay cannot open the file, it skips that sink and does not retry automatically. Records for that sink are lost. Fix the file access problem, then restart Relay to try again. Refer to File Sink Startup Failure Policy.

Update Rust Logging Configuration

If you create FileLogSinkConfig directly in Rust, add the failure_policy field. Set it to FileLogSinkFailurePolicy::Required to keep the default behavior. You can also use ..Default::default() to fill in missing fields. Existing TOML files need no changes to keep sinks required. Refer to Rust Library API.

Review Adaptive Hint Injection

Adaptive hints now use the settings in inject_header and inject_body_path. Previously, Relay always added the header and wrote body hints to nvext.agent_hints. Check settings that disable the header or choose another body path. These settings can now change the request sent to the provider.

Separate body path keys with dots, such as nvext.agent_hints. Do not leave empty keys, as in nvext..agent_hints. The component’s validation policy controls how Relay reports invalid paths. Relay creates missing objects along the path. If a parent value is not an object, Relay leaves the body unchanged. If the path points to metadata, Relay keeps the caller’s other metadata fields. Refer to Adaptive Hints.

Runtime and Plugin Reliability

Configure Python Object Decoding

BestEffortAnyCodec() now returns tagged dataclass and Pydantic values as JSON dictionaries by default. This also applies to values encoded earlier in the same process.

To restore objects, pass the expected classes in allowed_types. For example, use BestEffortAnyCodec(allowed_types=[SearchArgs, SearchResult]). Each codec has its own list. Encoding a value does not add its type to the list.

During decoding, the codec uses only classes in this list. It ignores old runtime type tokens and never imports modules. Restoring an object still runs its constructor or validators on the input data.

Use JsonPassthrough for tool arguments from a model. Use DataclassCodec or PydanticCodec for one known schema.

For LangChain structured responses, declare the expected classes in the request’s response_format.

For a complete example, refer to Using Codecs.

Update Streaming Callers

Node.js streaming wrappers now report provider and collector failures to the caller. A provider failure can reject stream creation or stream.next(). A collector that throws or rejects stops the stream. A failed finalizer produces a null aggregate, and stream.close() reports the failure.

Review code that treated every stream ending as success. Always await stream.close() in cleanup when you stop reading early. The provider receives an AbortSignal and must stop when it is aborted. Close waits for producer cleanup without a timeout. Collectors and finalizers can now return promises. Refer to Streaming LLM Execution.

Rust streaming parameters now use LlmCollector and LlmFinalizer for their callback fields. The builder accepts existing synchronous closures. For direct struct construction, convert synchronous callbacks with .into(). Set borrowed_collector to None unless you use the new borrowing callback. Refer to Borrowing Stream Chunks in Rust.

Check Dynamic Plugin Compatibility and Trust

The native host now provides application binary interface (ABI) v8. Its v7 callback layouts remain compatible. Scope snapshots require v8. Worker snapshots require the new grpc-v1 snapshot extension. Both SDKs report an unsupported operation on older hosts. Check the host version before adopting snapshot APIs. Refer to Native ABI Reference and Scope Snapshot Extension.

Native plugin diagnostics uses an optional table after the ABI v8 prefix. The execution ABI and compat.native_api = "1" stay unchanged. Check diagnostic capabilities before using the extension. If your plugin requires it, set a Relay compatibility range that includes its availability. Refer to Native Diagnostics Extension.

Failed integrity or signature checks now always prevent plugin activation. An optional plugin is skipped. A required plugin stops host startup. Check plugin reports and correct artifact digests, signatures, or trusted keys before restarting.

Embedded hosts load native plugins from private copies of the verified bytes. These plugins must not rely on adjacent files at the loaded library’s location. The CLI keeps its activation snapshot and adjacent native dependencies. Native library digest checks reject symbolic links and special files. Refer to Package Discoverable Plugins.

Short worker callbacks still have a 30-second default deadline. Use load.callback_timeouts in the plugin manifest to change the default or set a deadline for one callback. The host sets no deadline for execution callbacks. When a worker call ends, Relay cancels its active provider futures and streams. Check custom providers that need separate cleanup for background work. Refer to Worker Middleware and Continuations.

Review Worker Message Limits

Relay now limits the size of messages sent to and received from workers. The default is 4 MiB per message. Check large histories, tool results, and stream chunks against this limit. It does not limit the total stream size or change the model’s context limit.

Set the default with load.max_message_bytes.default_bytes in relay-plugin.toml. To set a limit for one callback, add its exact registration name under load.max_message_bytes.by_name. Host environment settings take priority over the manifest.

Upgrade the host and worker SDKs together before setting limits by callback name. Reactivate the plugin after changing settings. Custom workers must include the callback name in gRPC metadata so Relay can apply its limit. Refer to Configure Execution Message Size.

Handle Worker SDK Errors

Update Rust matches that list every WorkerSdkError variant. Handle the new Downstream { source, continuation_id } variant. Inspect source for the original error, and propagate the full error with ? when you do not handle it.

Python downstream errors can now be subclasses of WorkerSdkError. Use except WorkerSdkError or isinstance instead of exact-type checks. Re-raise the original exception when passing it on.

Preserving the original error keeps downstream failures from being counted as separate plugin callback failures. Refer to Rust Worker SDK and Python Worker SDK.

Related Release Information

For release highlights, compatibility updates, and current known issues, refer to the Release Notes. Use GitHub Releases for the complete release history and notes for a specific tag.