Migration Guides

Use this page to upgrade from NeMo Relay 0.9.0 to 0.10.0.

These steps also cover configuration and telemetry changes from the 0.9 patch releases. Apply them when upgrading directly from 0.9.0.

Upgrade to NeMo Relay 0.10.0

Update Packages and Plugin SDKs

Update the Relay CLI, application bindings, and plugin SDK dependencies to 0.10.0. Rebuild Rust applications and native or worker plugins against the updated dependencies. If you use the OpenClaw plugin, update its package too. It requires the matching Node.js binding version.

The optional Python LangChain NVIDIA integration now requires langchain-nvidia-ai-endpoints>=1.4.2,<2.0.0. Update older version requirements and regenerate the environment’s lockfile before installing the integration. For package names and installation commands, refer to Installation.

Move Primary Daemon Logging Settings

Move the daemon service’s logging settings to [daemon.logging]. Keep [logging] for other Relay processes, such as agent launches. CLI options and NEMO_RELAY_LOG* environment variables still override TOML settings for both. The following example enables JSON Lines logging to standard error:

[daemon.logging]
level = "info"
stderr_enabled = true
stderr_format = "jsonl"

For the order in which Relay applies logging settings, refer to Operational Logging.

Require Observability Configuration Version 4

Change version = 3 to version = 4 inside the observability component’s configuration, including programmatic configuration objects. Keep the existing typed trace endpoints. Adding log or metric sections is optional. The outer plugin configuration remains version = 1. The following example uses observability version 4 with a trace endpoint:

version = 1
[[components]]
kind = "observability"
enabled = true
[components.config]
version = 4
[[components.config.opentelemetry.endpoints]]
type = "gen_ai"
endpoint = "http://localhost:4318/v1/traces"

For the complete schema, refer to Observability Configuration.

Review Automatic OTLP Export and Collector URLs

OpenTelemetry Protocol (OTLP) endpoint environment variables now enable automatic export when their values are not empty or whitespace. The generic OTEL_EXPORTER_OTLP_ENDPOINT enables traces, logs, and metrics. Each endpoint variable for a specific signal enables only that signal. Automatic traces use the gen_ai projection, which maps Relay events to GenAI attributes. Relay can also export to its configured destinations, so existing settings can produce extra exports after the upgrade.

Review the launch environment and choose the exports you need. Remove unintended endpoint variables. To disable automatic and configured OTLP export, set opentelemetry.enabled = false in the observability component. To disable every Relay OTLP exporter, use OTEL_SDK_DISABLED=true. Resource and batch settings alone do not enable export.

If you configure Relay endpoints, move the process’s global OTLP header variables to each endpoint’s headers, header_env, or header_file settings. Relay skips a configured endpoint if global OTLP headers are set when it starts. The underlying exporter cannot keep those credentials separate. Automatic exporters can still use the global header variables.

Use the collector’s final URL. All OTLP/HTTP exporters now reject redirects, including log and metric exporters without headers. Remote automatic trace endpoints require HTTPS. Automatic log and metric endpoints with headers also require HTTPS. Plain HTTP remains available for localhost and loopback IPs. For supported variables and the order in which Relay applies them, refer to Automatic OpenTelemetry Configuration.

Update LLM Execution Intercepts

Relay 0.10.0 changes every large language model (LLM) execution-intercept callback, including streaming callbacks. Update their arguments as follows:

The new context identifies the selected request codec. A codec converts data between a provider’s format and Relay’s common format. When a request codec is available, the context provides decode and encode operations. Non-streaming execution also identifies the response codec and provides decoding. Streaming execution has no response codec because chunks are not complete provider responses.

Codec access lasts only for the current call. Do not store the context or selected codec for later calls. Non-streaming access ends when the callback finishes. Streaming request access ends when the returned stream closes.

Existing execution-intercept code and compiled plugins need updates. Make the following changes:

• Update callback argument order in the bindings you use. Recompile affected applications that use compiled bindings.
• Rebuild every native plugin against the 0.10 SDK. Native plugins continue to declare compat.native_api = "1", but must set compat.relay to ">=0.10.0,<1.0" or another range that excludes Relay 0.9. Relay 0.10 uses version 7 of the internal application binary interface (ABI). This interface defines how compiled plugins call the host. Relay rejects compiled layouts from versions 2 through 6.
• Regenerate and rebuild gRPC workers that register an LLM execution intercept. The protocol remains grpc-v1, but those workers must also exclude Relay 0.9 in compat.relay.

Raw native asynchronous intercepts also need the registration APIs for LLM execution. Replace non-streaming LLM execution uses of PluginContext::register_async_middleware_raw with register_async_llm_execution_intercept_raw, backed by the ABI v7 plugin_context_register_async_llm_execution_intercept host function. Use register_async_stream_middleware_raw for streaming execution. Its callback now receives codec context. The generic asynchronous middleware registration rejects both LLM execution kinds.

Raw asynchronous streaming code must keep a reference to the output stream while it uses the request codec. Use the stream’s decode and encode operations. Release the reference after its last use. The typed SDK manages these references for its codec wrappers.

LLM request intercepts and conditional middleware do not change. For the full context and lifetime contract, refer to Build Plugins, Native ABI Reference, and gRPC-v1 Worker Protocol.

Update Custom Worker Protocol Bindings

The worker protocol remains grpc-v1. Regenerate custom bindings. Rust code that lists every field in a protobuf message literal must add the following fields. Use generated defaults if you do not need the new values. The following table lists each field and its value when unused:

Custom implementations of the RelayHostRuntime service must also implement the new Log remote procedure call (RPC). The PluginWorker service does not gain a method. Workers that need logging through the host or earlier scope timestamps must declare a compat.relay range beginning at 0.10.0. Older hosts can lack Log or ignore the optional timestamps. Ordinary SDK scope calls still use the host’s current time. Refer to gRPC-v1 Worker Protocol.

Review Python Exception Handling

In Python, managed LLM and tool calls and intercepts now raise the original callback exception. The exception keeps its type, attributes, and traceback, which shows where the error occurred. If your error handlers catch only RuntimeError for callback failures, update them to catch the exceptions that your callbacks raise. Runtime failures and guardrail rejections still raise RuntimeError. This change does not apply to worker plugins.

Callbacks and stream iteration preserve asyncio.CancelledError. If the iterator raises it during aclose() cleanup, Relay reports RuntimeError. Handle cancellation separately from ordinary exceptions when needed. A streaming callback that returns an object that is not an async iterator raises TypeError.

Review Linux CLI Requirements

The musl CLI binaries now require musl 1.2 or newer on the host to load native plugins. Install the musl runtime, which includes its loader and C library (libc). Earlier musl binaries included the C library and could run without this runtime. If you used those binaries on a glibc-based system, switch to the GNU binary or install the required musl runtime.

The Unix shell installer now selects the GNU binary when it detects glibc. Otherwise, it selects the musl binary. Refer to Installation for target names and requirements.

Update Rust and Go Struct Literals

The following public Rust types gained fields after 0.9.0. If your code lists every field in a struct literal, add these fields. Use builders or defaults where the type provides them. The following table lists the added fields:

The Rust enum variant FlowError::CallbackException also gains a source field. When creating this error without an underlying exception, add source: None. When matching this error in Rust, include source or use .. to ignore fields that you don’t need.

In Go observability or propagation struct literals, use field names if your code currently lists values without names. Saved configuration and propagation payloads remain valid when they omit the new optional fields. The propagation wire version remains 1.

Set promote_resource_metadata_prefixes to an empty list when you do not want log or metric endpoints to copy root-scope metadata into resource attributes. An omitted field defaults to an empty list in TOML and JSON. Log and metric endpoints derived from a trace endpoint inherit its setting. Choose metadata values from a small, controlled set to limit the number of resources. For setting priorities and resource limits, refer to OpenTelemetry.

Preserve Trace Context Across Boundaries

Send the complete captured propagation context when continuing work in another scope stack or process. This context links work across boundaries. Its optional W3C fields now preserve an imported upstream trace, sampling flags, and vendor state. Sampling flags show whether the upstream trace was sampled. A valid imported W3C parent overrides trace IDs that Relay derives, including a supplied explicit root. Relay ignores malformed W3C values and continues with valid Relay propagation data.

Capture without a root removes the trace derived from a local Relay sender. It keeps a previously imported W3C parent. If you always need a new OpenTelemetry trace, remove traceparent and tracestate from that context. PropagationContext.to_traceparent() now forwards a valid imported parent unchanged. Capture and import context before leaving a trace if later calls must continue it. A scope handle outside the current stack does not keep its earlier W3C context. Refer to Scope Concepts.

Managed LLM execution replaces Relay’s traceparent and tracestate headers on provider requests. Import the upstream propagation context to preserve that parent instead of setting those headers manually. Capture the managed LLM context inside its provider callback if you need that span as a later parent. After the call returns, capture uses the enclosing active scope or imported context. The completed LLM span is no longer current.

Review Token Usage and Cost Reports

The optional Usage.uncached_input_tokens count is present only when the provider reports how many input tokens were not cached. Do not treat an absent value as zero. Anthropic compaction usage that combines multiple models no longer receives a model-based cost estimate. Review cost reports that assumed every response could be priced from its response model. Refer to Provider Response Codecs.

Update GenAI Attributes and Metric Dimensions

The GenAI projection replaces the old attribute gen_ai.usage.cache_creation.input_tokens with gen_ai.usage.cache_write.input_tokens. Update collector queries, dashboards, and alerts that use the old attribute name.

The projection also no longer adds nemo_relay.session.instance_id to metric attributes. Update dashboards that group or filter on this session attribute. Matching measurements now combine into the same time series across sessions.

Review GenAI Tool Content and Completion Marks

GenAI tool spans now preserve every JSON argument and successful result except null, after sanitizers process the content. They use a consistent JSON text format, including for arrays, strings, numbers, and booleans. Update consumers that assumed all values were objects. Review collector access and sanitizers for the newly kept content. enable_full_payloads = false does not disable tool-content export. For the projection contract, refer to GenAI Projection.

Managed streaming telemetry adds gen_ai.response.time_to_first_chunk and the gen_ai.client.operation.time_to_first_chunk histogram in seconds. These measure the first provider protocol chunk, not the first text token. Keep those measurements separate in latency dashboards.

Coding-agent end hooks without a matching start now produce agent_end_without_start, turn_end_without_start, subagent_end_without_start, or tool_end_without_start marks. Consumers must allow start_observed = false, absent start times and durations, and an absent parent when no existing parent can be verified. ATIF and the GenAI projection omit these partial observations. Use ATOF, the full OpenTelemetry projection, or OpenInference when you need them.

Relay reserves tool_end_without_start. Any application or plugin that emits that mark must provide a string in data.tool_name that is not empty or whitespace. Otherwise, emission fails. Its arguments and result now pass through tool sanitizers before event sanitizers. Rename any unrelated custom mark that used that name. Refer to Completion Hooks Without a Start.

Refresh Relay-Managed Coding-Agent Integrations

After upgrading the CLI, run the following command to refresh personal Relay-managed coding-agent integrations before starting new sessions:

nemo-relay integrations refresh

On Windows, this command updates installed Claude Code hooks to use paths that work in Bash. It also updates Codex hooks to use PowerShell commands that preserve input, output, and exit codes. Existing installed hooks keep their old commands until you refresh them.

To refresh only Codex, use nemo-relay install codex --force. Refreshing Codex switches persistent routing to the built-in OpenAI provider. This preserves session identity and routes image generation through Relay. It replaces URLs that contain credentials with a plain gateway URL. It stores managed authentication in $CODEX_HOME/.env. Quit and restart Codex after refreshing to load the header. The gateway still accepts older capability URLs for older clients.

For daemon bundles managed by an administrator, follow the daemon upgrade workflow. Close sessions and let workers finish pending requests. Then stop the service and replace the binary at the same path. Keep the saved identity state and existing bundle files.

Update Codex Trajectory Consumers

Codex now exports one Agent Trajectory Interchange Format (ATIF) file for each codex-task instead of one file for the whole session. A task groups parent turns and delegated agent work. Relay writes the file when the task closes. Separate tasks in the same Codex session produce separate files and traces.

If you use the default filename template, update file lookups from nemo-relay-atif-<session-id>.json to nemo-relay-atif-<task-scope-id>.json. Use extra.nemo_relay.session_id in each trajectory to group tasks from the same Codex session. Update tools that expect a cumulative file after each parent Stop hook. A task can stay open while delegated agents finish and the parent prepares its final response. Refer to Codex.

Review Managed Provider Credentials

Managed Codex routes now forward ChatGPT OAuth to the native ChatGPT Codex backend when no administrator OpenAI credential is configured. If the daemon has openai_auth_header or OPENAI_API_KEY, Relay replaces the incoming OAuth credential and uses the administrator’s configured OpenAI-compatible upstream. ChatGPT voice calls to POST /backend-api/codex/realtime/calls are an exception. They keep the caller’s ChatGPT credential and use the native ChatGPT backend even when a replacement credential is configured. Review this exception if your deployment relies on administrator credentials to select the upstream. Review those settings to choose the intended provider and credential source. Refer to Codex Responses Compatibility.

Managed Claude model routing requires an Anthropic API key or an administrator’s Anthropic-compatible gateway credential. Claude.ai OAuth is not supported with ANTHROPIC_BASE_URL. If your deployment uses only OAuth, keep native routing instead of setting the managed base URL. Refer to Daemon Configuration.

Choose Managed Daemon Startup Policy

Model Context Protocol (MCP) initialization can now finish after the client authenticates and registers. A worker starts in the background. By default, authenticated requests use passthrough during startup and recovery. Passthrough forwards requests without worker plugins. New requests switch to the worker after runtime, plugin, control, and HTTP readiness checks pass.

If every request must use worker guardrails, plugins, or observability, add --require-worker to the daemon service command. Configure clients to handle HTTP 503 while a worker is unavailable. This flag conflicts with --pass-through, which disables workers. Neither /healthz nor an MCP handshake proves that worker policies are active. Verify worker status and an observable plugin result.

Worker startup has no deadline or retry limit. Update monitoring that assumed a fixed startup timeout. Check worker_startup_pending, worker_added, and worker_activation_failed for progress, elapsed time, and retry delays. Retry delays increase up to 60 seconds. Active requests and streams keep their original destination during route changes. Refer to Daemon Operations.

On Windows, managed workers request breakaway when the host Job Object permits it. Otherwise they start inside the host job and emit worker_lifetime_limited. This respects the host’s job policy. Closing that host can stop the worker and require route reactivation after the broker’s recovery grace period. For process ownership and log destinations, refer to Register the MCP Lifecycle Client.

Revalidate Shared Python Worker Environments

Older managed Python environments without .nemo-relay-environment.key now require the original installer’s bootstrap key to verify their signed installation records. An environment that previously validated without that key can fail validation or startup after upgrading. If the original key is missing or unreadable, recreate the environment from the trusted plugin source. Newly provisioned environments keep their verification key with the environment.

For system-installed workers, make sure each daemon user can read the configuration, manifest, lifecycle state, and environment. Users must also be able to access their directories. Only administrators should have write access. Lifecycle state records the plugin’s installation and validation status. Startup and inspection validate shared state without rewriting it. Run nemo-relay plugins validate <plugin-id> to check the installation. This command saves its validation result. Refer to Discoverable Plugins.

Review Anthropic Compaction Cache Support

Compaction summarizes older conversation content to reduce its size. Relay 0.9.1 bypassed the response cache for all Anthropic compaction traffic. Relay 0.10.0 can cache validated paused-threshold and on-demand compaction responses and their supported continuations with anthropic-version: 2023-06-01 and the matching compaction beta headers. Default non-paused threshold compaction and unsupported or malformed traffic still run live.

No new cache setting is required. The existing rules for requests that can produce different results still apply. Requests without explicit temperature = 0 need cache_nondeterministic = true to be eligible. Review expected cache hits and anthropic_compaction bypass marks. Compare them with the supported cases in LLM Eligibility and Limits.

Related Release Information

For release highlights, compatibility updates, and current known issues, refer to the Release Notes. Use GitHub Releases for the complete release history and notes for a specific tag.