> This page is for NeMo Relay, version v0.8.4.
> For other versions, use one of these documentation indexes:
> - Latest (v0.10.0) (default): https://docs.nvidia.com/nemo/relay/latest/llms.txt
> - dev: https://docs.nvidia.com/nemo/relay/dev/llms.txt
> - v0.10.0: https://docs.nvidia.com/nemo/relay/v0.10.0/llms.txt
> - v0.9.4: https://docs.nvidia.com/nemo/relay/v0.9.4/llms.txt
> - v0.8.4: https://docs.nvidia.com/nemo/relay/v0.8.4/llms.txt
> - v0.7.3: https://docs.nvidia.com/nemo/relay/v0.7.3/llms.txt
> - v0.6.0: https://docs.nvidia.com/nemo/relay/v0.6.0/llms.txt
> - v0.5.0: https://docs.nvidia.com/nemo/relay/v0.5.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/relay/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/relay/_mcp/server.

# Migration Guides

> Plan an upgrade from NeMo Relay 0.7 to 0.8.

Use this page to plan an upgrade from NeMo Relay 0.7 to 0.8. It groups the
actions from the release notes by the surface you operate. If you skip one or
more releases, review the migration guides and release notes for every
intervening release in sequence.

## Upgrade to NeMo Relay 0.8

### Refresh Relay-Managed Coding-Agent Integrations

After upgrading to 0.8.2 or later, refresh every Relay-managed Codex and Claude
Code integration before starting a new session:

```bash
nemo-relay integrations refresh
```

The command retires old MCP generations and replaces Relay-owned sidecars with
the current Relay binary. It does not modify manually configured MCP servers.
For a focused repair of one managed Codex installation, use
`nemo-relay install codex --force`. This refresh also replaces the older
uninstall-and-install procedure for the persistent Codex provider configuration
needed by the initial 0.8 release.

### Update Exhaustive Cache Configuration Literals

Rust code that constructs `ResponseCacheConfig` with an exhaustive struct
literal must add `tools: None`. Prefer `..ResponseCacheConfig::default()` when
the literal should remain compatible with new optional cache surfaces.

### Stream Node.js LLM Execution Intercepts Lazily

> **Breaking Change**
>
> NeMo Relay 0.8 changes Node.js LLM streaming execution intercepts from buffered
> JSON values to lazy async iterables. `next(request)` now resolves to an
> `AsyncIterable<Json>`, and the intercept callback must return an async iterable
> or a promise of one. Scalar and array return values are rejected.

This change applies to global, scope-local, and plugin-owned registrations made
with `registerLlmStreamExecutionIntercept`,
`scopeRegisterLlmStreamExecutionIntercept`, and
`PluginContext.registerLlmStreamExecutionIntercept`.

Replace array transforms such as this pre-0.8 callback:

```js
async (request, next) =>
  (await next(request)).map((chunk) => ({ ...chunk, intercepted: true }));
```

Use an async generator to transform downstream chunks incrementally:

```js
async function* transformStream(request, next) {
  for await (const chunk of await next(request)) {
    yield { ...chunk, intercepted: true };
  }
}
```

Return `next(request)` directly when the intercept only forwards the stream. To
short-circuit the downstream chain, return an async generator that yields the
replacement chunks. Do not collect the iterable into an array unless the
application intentionally needs to buffer the complete response.

### Move Hermes Agent to Its Native Relay Integration

NeMo Relay 0.8 removes Hermes Agent from the Relay CLI. The `nemo-relay hermes`
shortcut, `nemo-relay run --agent hermes`, Hermes install, uninstall, doctor,
configuration, MCP selection, hook forwarding, and `/hooks/hermes` endpoint are
no longer supported. Relay configuration under `[agents.hermes]` is also
unsupported.

Remove the `[agents.hermes]` section from every Relay configuration file before
upgrading. If the section remains, Relay rejects the configuration and prevents
configuration-resolving commands, including Claude Code and Codex runs, from
starting.

NeMo Relay is built into Hermes Agent. Do not install Relay separately or
enable an observability plugin. Hermes Agent understands NeMo Relay plugin
configurations.

### Migrate to the Switchyard-Owned Dynamic Plugin

Support for the Switchyard integration now shifts to the
[Switchyard project](https://github.com/NVIDIA-NeMo/Switchyard). Switchyard
0.3.0 will provide the dynamic plugin, release artifacts, and configuration
documentation.

NeMo Relay >=0.8.0 no longer ships the experimental `nemo-relay-switchyard`
crate, the CLI `switchyard` feature, or the service-backed `switchyard`
component. Configurations that contain `[[components]]` entries with
`kind = "switchyard"` are rejected with migration guidance.

To migrate an existing configuration:

1. Remove the CLI `switchyard` Cargo feature and any dependency on
   `nemo-relay-switchyard`.
2. Remove the legacy `[[components]]` entry with `kind = "switchyard"`.
3. Remove Decision API settings and ATOF HTTP sinks used only by the former
   service integration. Keep any ATOF sink that another Relay consumer uses.
4. Install and configure the Switchyard-owned dynamic plugin by following the
   documentation published with Switchyard 0.3.0.

Do not copy service-era component fields into a dynamic-plugin entry. The
Switchyard release documentation is the source of truth for supported
algorithms, target configuration, and migration details. For Relay's manifest,
trust, policy, and activation model, refer to
[Configure Discoverable Plugins](/configure-plugins/discoverable-plugins).

### Return Canonical Tool Execution Results

> **Breaking Change**
>
> NeMo Relay 0.8 replaces raw managed tool results with
> `ToolExecutionResult`. Update managed tool callbacks, execution-intercept
> continuations and outcomes, execute-helper consumers, and manual tool end calls
> as one cutover.

The canonical shape has a required application-owned `result` and an optional
opaque JSON `annotation`. Relay transports the annotation alongside the result
without interpreting it:

```json
{
  "result": {"hits": 2},
  "annotation": {"source": "cache"}
}
```

Update Python callbacks and read `.result` where the application needs the
business payload:

```python
async def search(args):
    return nemo_relay.ToolExecutionResult(
        {"hits": 2},
        {"source": "cache"},
    )

execution_result = await nemo_relay.tools.execute("search", {"query": "weather"}, search)
hits = execution_result.result["hits"]
```

Update Node.js callbacks to return the canonical object:

```js
const executionResult = await toolCallExecute(
  "search",
  { query: "weather" },
  async () => ({
    result: { hits: 2 },
    annotation: { source: "cache" },
  }),
);
const hits = executionResult.result.hits;
```

Update Rust callbacks to return `ToolExecutionResult`:

```rust
use nemo_relay::api::tool::ToolExecutionResult;

.func(Arc::new(|_args| {
    Box::pin(async {
        Ok(ToolExecutionResult::annotated(
            json!({"hits": 2}),
            json!({"source": "cache"}),
        ))
    })
}))
```

An execution intercept's `next(args)` now returns `ToolExecutionResult`.
Forwarding intercepts must preserve both fields. Rust can use
`ToolExecutionInterceptOutcome::from(result)`. Python, Node.js, Go, and public
C callbacks return their binding-specific canonical outcome with `result` and
`annotation`. Legacy raw callback and intercept returns are rejected.

Relay 0.8 establishes this result contract as the native API 1 and `grpc-v1`
baseline. Set `compat.native_api = "1"` or `compat.worker_protocol = "grpc-v1"`,
rebuild every dynamic plugin, and declare a `compat.relay` range that excludes
Relay versions before 0.8. Use `>=0.8.0,<1.0` unless you deliberately need a
narrower or open-ended 0.8-or-newer range. The manifest is the plugin author's
compatibility assertion, not proof that an artifact was rebuilt.

The native ABI remains v4. Workers continue to use `grpc-v1` and the
`nemo.relay.worker.v1` protobuf package, but this baseline changes the
`ToolNext` response from `JsonResult` to `ToolExecutionResultResponse` and
changes `ToolExecutionInterceptResult.outcome` from `JsonEnvelope` to
`ToolExecutionInterceptOutcome`. Regenerate worker protobuf bindings during
the required rebuild. Future incompatible contract changes must increment the
corresponding native API or worker protocol version.

Manual tool end helpers also require the canonical wrapper. For example, pass
`nemo_relay.ToolExecutionResult(payload)` to Python `tools.call_end`,
`{ result: payload }` to Node.js `toolCallEnd`, and
`ToolExecutionResult::new(payload)` to the Rust
`ToolCallEndParams.execution_result` builder field.

Tool sanitize-response guardrails continue to receive only the business
`result`. A successful tool end event stores a non-null annotation under
`category_profile.tool_result_annotation`, after which a scope-end event
sanitizer can replace or remove it. Refer to
[Tool Execution Intercept Outcomes](/reference/tool-execution-intercept-outcomes)
for the complete continuation, lifecycle, exporter, and binding contract.

### Upgrade Native Rust Plugins to NeMo Relay 0.8

> **Breaking Change**
>
> NeMo Relay 0.8 is not source-compatible with typed native Rust plugins built
> against NeMo Relay 0.7. Update every typed guardrail, sanitizer, and intercept
> callback to the asynchronous contract, rebuild the plugin with
> `nemo-relay-plugin` 0.8.0, and update its Relay compatibility range before
> deployment.

NeMo Relay 0.8 changes every typed native Rust guardrail, sanitizer, and
intercept callback to return a future. This is a source-breaking SDK change.
Subscribers, validation, registration, scope and mark helpers, raw synchronous
ABI callbacks, Go, and worker subscriber contracts remain synchronous.

Use owned callback arguments and return `Result<T>` from an async block:

```rust
context.register_tool_request_intercept("policy", 10, false, |_name, args| {
    async move {
        // Await policy I/O here.
        Ok(args)
    }
})?;

context.register_tool_execution_intercept("policy", 10, |_name, args, next| {
    async move {
        let result = next.call(args).await?;
        Ok(ToolExecutionInterceptOutcome::from(result))
    }
})?;
```

Event sanitizers now receive `Arc<Event>`. `ToolNext`, `LlmNext`, and
`LlmStreamNext` are owned, cloneable values with async `call` methods. Stream
interceptors return a future containing `LlmJsonAsyncStream`. LLM sanitizer
codec facades remain available before and after an await for the lifetime of
the middleware completion.

The SDK owns one multi-thread Tokio runtime for each configured plugin component,
starts it lazily, and uses two workers by default. Override
`NativePlugin::executor_config` to select a different nonzero worker count.
Use async I/O or `spawn_blocking`; do not block executor workers. Root
middleware futures preserve Relay scope context across awaits and worker-thread
migration. Child tasks created with `tokio::spawn` do not inherit that context
automatically.

Set new typed async native plugin manifests to:

```toml
[compat]
relay = ">=0.8.0,<1.0"
native_api = "1"
```

Relay 0.8's current native ABI is v4, with fallback to frozen v3 and v2 tables
for old binaries. ABI v4 includes typed asynchronous middleware operations such
as completion-scoped codecs and pull-based downstream LLM streams, then appends
`emit_mark_v2` for typed data-schema and log-severity options. This does not
change the manifest `native_api` value. `native_api = "1"` uses the canonical
`ToolExecutionResult` contract without changing the ABI v4 table layout. Rebuild
all native plugin artifacts; support for frozen table layouts does not preserve
the pre-0.8 JSON semantics.

### Move Project Configuration to a Supported Location

NeMo Relay 0.8 no longer discovers repository-local configuration. Files named
`.nemo-relay/config.toml`, `.nemo-relay/plugins.toml`, and
`.nemo-relay/.dynamic-plugins.json` do not affect normal commands, plugin
activation, or dynamic-plugin lifecycle state.

Move settings that should apply to your account into these files:

* `$XDG_CONFIG_HOME/nemo-relay/config.toml` and `plugins.toml`
* `~/.config/nemo-relay/config.toml` and `plugins.toml` when
  `XDG_CONFIG_HOME` is not set

Use `/etc/nemo-relay/config.toml` and `/etc/nemo-relay/plugins.toml` on Unix,
or `%ProgramData%\nemo-relay\config.toml` and
`%ProgramData%\nemo-relay\plugins.toml` on Windows, for system policy. System
configuration has higher precedence than the selected user or explicit
configuration.

For a deliberately selected file in any location, pass `--config` or
`--plugin-config-path`. A `plugins.toml` beside an explicit `config.toml` is
still selected automatically. Relay does not move, rewrite, or delete legacy
project files. Run `nemo-relay doctor` to find ignored files in the current
directory's ancestors.

Remove `--project` from config, plugin, and model-pricing commands. The
interactive `nemo-relay config` workflow now writes user configuration only,
and `config --reset` resets that user file. The former setup project/both
choices and `config --reset --scope` option have been removed. `--user` and
`--global` remain available.

Local output directories such as `.nemo-relay/atof`, `.nemo-relay/atif`, and
logs remain supported. Only configuration discovery and dynamic-plugin
lifecycle state lose project semantics.

## Related Release Information

For release highlights, compatibility updates, and current known issues, refer
to the [Release Notes](/about-nemo-relay/release-notes). Use [GitHub Releases](https://github.com/NVIDIA/NeMo-Relay/releases) for the complete
release history and notes for a specific tag.