> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/relay/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/relay/_mcp/server.

# Release Notes for NVIDIA NeMo Relay

> Review highlights, compatibility updates, fixed known issues, and current known issues for NVIDIA NeMo Relay 0.7.

This page contains the release notes for [NVIDIA NeMo Relay](/about-nemo-relay/overview).

## Release 0.7

NVIDIA NeMo Relay 0.7 adds asynchronous middleware, codec-aware observability
sanitization, typed OpenTelemetry exporters, cross-boundary context
propagation, and opt-in exact-match LLM response caching. It also expands CLI
configuration, packaging, logging, and diagnostics workflows.

### Highlights

The following highlights summarize the most significant user-facing changes in
0.7:

* **Asynchronous middleware and ordered event delivery:** Rust middleware
  callbacks can await work directly, Python callbacks can return awaitables,
  and Node.js callbacks can return Promises. Scope and mark APIs remain
  synchronous while Relay publishes sanitized events through an ordered
  background dispatcher.
* **Codec-aware privacy controls:** LLM sanitizers receive the active per-call
  codec across in-process, native-plugin, and worker boundaries. The PII
  redaction plugin can protect mixed OpenAI Chat, OpenAI Responses, and
  Anthropic Messages traffic without a fixed provider codec.
* **Unified OpenTelemetry export:** One typed, multi-endpoint configuration
  supports `full`, `gen_ai`, and `openinference` projections. The `gen_ai`
  projection implements a pinned
  [OpenTelemetry GenAI semantic-conventions v1.42-era snapshot](https://github.com/open-telemetry/semantic-conventions-genai/tree/43633a68ef8f8ed87a1d5eb205990311ca708bf1).
* **Portable Relay context:** Transport-neutral propagation can seed isolated
  scope stacks across process and runtime boundaries. Python applications can
  also fork the active scope stack for concurrent `asyncio` tasks.
* **Opt-in LLM response caching:** The Adaptive plugin can serve eligible
  exact-match calls from in-memory or Redis stores. The cache remains disabled
  unless `response_cache` is configured.
* **CLI and operational improvements:** The CLI adds structured configuration
  editing, PyPI distribution, size-based log rotation, and offline `doctor`
  probes. Python, Node.js, and Go bindings initialize default operational
  logging from the same environment controls as the CLI.

### Support Matrix and Compatibility Updates

LLM sanitizer callbacks require migration in 0.7. Callbacks now receive the
payload first and a directional context second, and they return an optional
payload. There is no one-argument compatibility adapter. Rust worker mark,
scope, tool, and LLM sanitizer callbacks are also asynchronous in 0.7. Native
plugins and workers must be rebuilt with the matching 0.7 SDK and protocol
definitions.

The [Support Matrix](/reference/support-matrix) is the canonical reference for
supported platforms and architectures, worker runtimes, coding agents, and
integrations. It also records current limitations, including platform-specific
worker requirements.

Before upgrading from 0.6, follow the
[0.7 Migration Guide](/reference/migration-guides#upgrade-to-nemo-relay-07).
For the new callback contract and codec operations, refer to
[Codec-Aware LLM Sanitizers](/about-nemo-relay/concepts/middleware#codec-aware-llm-sanitizers).

Rust callers that exhaustively match `FlowError` must handle the new
`CallbackException` variant. It retains Python and JavaScript exception types
for OpenTelemetry projection while preserving the existing `internal_error`
classification.

Observability configuration version 3 is a breaking change. It removes the
standalone `openinference` section and standalone OpenInference subscriber APIs,
and requires a type and endpoint for every OpenTelemetry exporter. Legacy mark
and attribute-alias controls remain available on typed `full` and
`openinference` endpoints. Refer to
[Migration Guides](/reference/migration-guides#observability-configuration-version-3)
for before-and-after configuration and binding API changes.

Each `opentelemetry.endpoints` entry can resolve sensitive header values from
environment variables through `header_env`. Plugin activation rejects missing
variables, blank values, and duplicate header names, then snapshots each valid
value for the endpoint.
Process-global `OTEL_EXPORTER_OTLP_HEADERS` and
`OTEL_EXPORTER_OTLP_TRACES_HEADERS` now fail construction or activation because
their values cannot be isolated between endpoints.

The experimental Relay-owned Switchyard plugin is deprecated in 0.7. It is
excluded from default CLI builds and is scheduled for removal in 0.8, when a
Switchyard-owned native plugin is expected to replace it.

### Fixed Known Issues in 0.7

The following known issues are fixed in 0.7:

* Sanitizer callback errors, panics, rejected awaitables or Promises, and
  invalid return values now fail closed. Relay withholds the governed
  observability payload instead of emitting the original unsanitized value.
* Relay removes known LLM credential headers before emitting request events,
  and the PII redaction plugin rejects malformed RFC 6901 `target_paths` before
  activation.
* Generated coding-agent enforcement hooks now fail closed for transport,
  startup, authentication, evaluation, malformed-response, and policy-delivery
  failures. Lifecycle and after-the-fact hooks continue to fail open.
* OpenTelemetry exporters now report finite batch-queue span loss through
  operational logs and plugin runtime diagnostics, including an exact dropped
  span count during graceful shutdown.
* Typed OpenTelemetry configuration now rejects colliding destinations after
  canonical endpoint resolution, validates attribute mappings before
  activation, and completes bare OTLP HTTP endpoints with `/v1/traces`.
* OpenTelemetry and OpenInference prefer response-side model attribution,
  preserve binding callback exception types, and map structured `FlowError`
  values to useful `error.type` attributes. OpenInference emits system content
  as ordered input messages.
* ATIF no longer duplicates user steps for same-turn continuations. Model
  fallback matches the span exporters, and metadata-routing and remote-delivery
  failures remain visible and recoverable through flush and teardown.
* Explicit configuration editor targets are preserved, file layers follow one
  documented priority model, disabled plugin entries do not contribute
  configuration, and programmatic component enablement takes precedence over
  discovered files.
* Library initialization reports inherited `plugins.toml` files without
  exposing their values. `nemo-relay doctor` reports invalid explicit and
  logging configuration, supports offline exporter validation, and reports
  effective upstream authentication presence without exposing credentials.
* Transparent gateways preserve non-success streaming provider responses and
  have bounded, interruptible shutdown behavior.
* Node.js preserves asynchronous propagation context, includes queued callback
  delivery in flush completion, and isolates subscriber exceptions instead of
  aborting the process.
* Python plugin teardown is safe from a running `asyncio` loop, cancelled
  scopes are reported as errors, and completion callbacks no longer target a
  closed event loop during cancellation.
* Prebuilt CLI binaries carry the intended release version, so an upgraded CLI
  does not silently reuse a gateway started by a differently identified build.
* `object_store` is upgraded to `0.14.1`, which removes the temporary
  `RUSTSEC-2026-0194` and `RUSTSEC-2026-0195` advisory exceptions.
* Embedded Python hosts can use
  `plugin.load_dynamic_plugin_activation_specs(path)` to convert the standard
  `[[plugins.dynamic]]` records in one explicitly selected `plugins.toml` into
  the activation specs accepted by `initialize_with_dynamic_plugins()`. This
  scoped 0.7 compatibility helper removes host-side TOML and manifest parsing;
  a future unified file-backed initializer is expected to replace it.
* Programmatically declared plugin components now apply their `enabled` value
  over discovered file configuration. When code re-enables a component that a
  discovered file disabled, initialization reports a warning that names the
  component and source file.
* Library plugin initialization now emits a warning for each discovered
  `plugins.toml` file inherited by the caller configuration. The warning is
  available in operational logs, initialization results, and the active plugin
  report without exposing configuration values or credentials.
* LLM payload redaction now follows the codec active for each call instead of a
  codec captured from plugin configuration. Codec-dependent policies omit the
  observability payload and annotation when Relay cannot safely normalize the
  payload. This change resolves the mixed-provider PII sanitization issue
  [#526](https://github.com/NVIDIA/NeMo-Relay/issues/526).

## Known Issues in 0.7

* Go and the raw C FFI remain experimental and source-first. Generated API
  pages focus on Rust, Python, and Node.js.
* Local coding-agent observability depends on host hooks and provider traffic
  reaching the local gateway. Relay cannot fully capture remote or cloud
  execution that bypasses the local host.
* Persistent Codex, Claude Code, and Hermes integrations use user-scoped
  configuration and a shared loopback gateway. Use `nemo-relay run` when a
  launch must retain project-specific configuration.
* On Windows, a restrictive host Job Object can limit gateway reuse or prevent
  persistent bootstrap. Codex can also make a cold-start `/models` request
  before required MCP servers start; Relay retries the request.
* Codex 0.143 does not expose `SessionEnd`, and Codex multi-agent v2 encrypts
  delegated-task payloads that Relay cannot decrypt or reliably link.
* The Node.js binding and package workflows require Node.js 24 or later.
* The deprecated Switchyard plugin is experimental, excluded from default CLI
  builds, and requires a separately managed compatible service and a named
  authenticated ATOF HTTP stream sink. It is scheduled for removal in 0.8.
* OpenClaw has public hook-backed telemetry. Its security and optimization
  coverage is partial because it does not own a managed execution path.
* The NeMo Guardrails remote backend inherits its configured service's
  availability, latency, and policy behavior. The local backend requires Python
  3.11 or later and `nemoguardrails==0.22.0`.
* The PII redaction plugin currently supports its deterministic local backend;
  local-model backend configuration is reserved for future work.
* Pricing and optimization estimates depend on model names, token data, pricing
  sources, and freshness evidence. Missing or inconsistent evidence produces
  partial or absent cost fields rather than zero values.
* ATOF stream sinks and remote ATIF storage require reachable, correctly
  configured destinations. A failed stream sink does not stop file output or
  other active sinks.
* ATIF omits point-in-time marks. Use ATOF for the canonical mark stream;
  the `gen_ai` OpenTelemetry projection also omits marks. The `full` and
  `openinference` projections retain their fixed native mark handling.
* Native dynamic plugins run in the Relay process and are not sandboxed. A
  `grpc-v1` worker runs in a separate process, but that process is not a
  security sandbox.
* Treat Python `LLMRequest` objects as immutable. Request middleware that
  changes content must return a new request object.
* Native subscriber callbacks arrive asynchronously. Flush subscribers before
  depending on their side effects, captured events, files, or exporter output.
* OpenTelemetry endpoints use finite batch queues. A burst that fills an
  endpoint queue drops completed spans without applying backpressure and can
  leave an incomplete trace, including a missing root span. The SDK warns on
  the first drop and reports the exact dropped-span count during graceful
  shutdown. Increase `OTEL_BSP_MAX_QUEUE_SIZE` before plugin activation to
  reduce the risk for a known burst size; endpoint-specific batch sizing is not
  available, and a larger finite queue does not guarantee lossless telemetry.
* Operational logging configuration and sink lifecycle are available, but broad
  operational log coverage across commands is not yet available.

## Previous Releases

For previous release notes, release artifacts, and the complete PR-by-PR
history, refer to [GitHub Releases](https://github.com/NVIDIA/NeMo-Relay/releases).

## Related Topics

* [Support Matrix](/reference/support-matrix)
* [Migration Guides](/reference/migration-guides)
* [Installation](/getting-started/installation)
* [NVIDIA NeMo Relay CLI](/nemo-relay-cli/about)
* [Known Issues and Troubleshooting](/resources/support-and-faqs)