NeMo Relay and Streaming Dependencies
Install NeMo Relay
NeMo Relay integration occurs in two ways:
The required integration depends on the agent harness. Relay-enabled runs with
the Claude, Codex, and Pi adapters require the
nemo-relay CLI.
NVIDIA NeMo Fabric 0.7 uses these Relay versions:
NeMo Relay Python Package
The root relay extra installs a version of the NeMo Relay Python package known
to be compatible with the installed version of NeMo Fabric:
LangChain Deep Agents, Hermes Agent, mini-SWE-agent, and NOOA import the NeMo
Relay Python package directly. Their adapter packages provide relay and
full extras:
Use relay when the environment already manages the harness. For Deep Agents, mini-SWE-agent, and NOOA, use full to install the harness and Relay package together. Hermes Agent’s relay and full extras install Relay. Install Hermes Agent separately.
Hermes v0.22.7 requires nemo-relay>=0.11,<0.12; Hermes v0.22.5 requires nemo-relay>=0.10,<0.11. The Hermes adapter supports both ranges. Use Python 3.14 for Hermes. Applications on Python 3.13 or another supported version can select a separate Hermes interpreter through ADAPTER_PYTHON.
Deep Agents, mini-SWE-agent, and NOOA require nemo-relay>=0.11,<0.12. Claude and Codex launch the external Relay CLI. Their harness and full extras install the CLI.
NeMo Fabric uses runtime-owned Relay configuration. Remove or relocate a user plugins.toml or the nearest .nemo-relay/plugins.toml before starting a Relay-enabled Deep Agents, Hermes Agent, or mini-SWE-agent runtime. Administrators manage system configuration in /etc/nemo-relay/plugins.toml.
Upgrade to Relay 0.11
The nemo-fabric[relay] extra requires Relay 0.11. Claude, Codex, and Pi support stable CLI releases from 0.9 through 0.11. The Pi extension must match the selected CLI release. The insecure_http_model_hosts setting requires CLI 0.11. Hermes v0.22.7 uses Relay 0.11; v0.22.5 remains supported with Relay 0.10. Restart existing runtimes after the upgrade so each gateway and extension uses the selected release.
Relay 0.11 gateways require HTTPS for remote model endpoints by default, including private network and Kubernetes Service addresses. Plain HTTP remains available for loopback endpoints. To trust a credential-free remote HTTP model endpoint, set harness.settings.insecure_http_model_hosts to its exact hostname or IP address. Prompts and responses travel unencrypted, and Relay strips credential headers before forwarding. Relay 0.11 also requires HTTPS for remote telemetry collectors by default. Relay supports explicit exceptions for trusted collectors through insecure_http_hosts, or NEMO_RELAY_OTEL_INSECURE_HTTP_HOSTS for automatic OTLP exporters. Relay releases before 0.10.2 can forward credentials to remote HTTP model endpoints. Use HTTPS when model requests or telemetry contain credentials or sensitive content.
NeMo Fabric generates a PreToolUse check for Claude and Codex. With Relay 0.10.1 and later, a delivery failure blocks the operation with exit code 2. Relay 0.9.x and 0.10.0 return exit code 1 for delivery failures.
Claude hooks set onFailure: "block" so delivery errors, process launch failures, crashes, and timeouts block the tool call. This requires Claude Code 2.1.295 or later; the pinned claude-agent-sdk==0.2.166 bundles 2.1.299.
NeMo Fabric wraps Codex PreToolUse checks to return exit code 2 with a blocking reason when Relay fails to start, times out, or returns a nonzero exit code. This also handles exit code 1 from Relay 0.9.x and 0.10.0. The wrapper has a 35-second timeout. Codex 0.154.0 can still continue if the wrapper itself fails to start or reaches the outer harness timeout.
Lifecycle hooks allow execution to continue when telemetry delivery fails. Guardrail denials return exit code 2 under both policies. The harness hook timeout is 45 seconds, which leaves a margin around Relay’s 30-second delivery timeout.
Migrate Relay Observability Configurations to Version 4
NeMo Fabric writes and accepts observability configuration version 4 and
rejects version 3. Relay 0.11 requires version 4. Change version = 3 to version = 4 in each observability
component configuration, including RelayObservabilityConfig(version=...) and
relay.components[].config.version. Typed OpenTelemetry endpoints keep the
same shape, and the outer plugin configuration stays version = 1. Omitting
the observability version selects version 4.
Relay also enables automatic OTLP export when an
OTEL_EXPORTER_OTLP_ENDPOINT or signal-specific OTLP endpoint variable is set
in the runtime environment, and it skips configured endpoints while global
OTLP header variables are set. Automatic traces use the gen_ai projection,
which exports prompt and response content. Remove unintended OTLP variables
from the environment that starts the runtime, set
opentelemetry.enabled = false in the observability configuration, or set
OTEL_SDK_DISABLED=true to turn off every Relay OTLP exporter. Refer to the NeMo Relay
migration guide
for the complete list of 0.11 changes.
Configurations from Relay 0.6 must also move flat OpenTelemetry exporter fields
into entries under opentelemetry.endpoints. In the Python SDK, replace the
removed RelayOtlpConfig model with RelayOpenTelemetryConfig and one or more
RelayOpenTelemetryEndpointConfig entries.
NeMo Relay CLI
Claude and Codex harness extras install their compatible CLI binary without
the Relay Python package. Pi requires a separate CLI installation alongside
the npm adapter. Install the >=0.9,<0.12 range:
If cargo-binstall is available, install the pinned CLI with Cargo:
After installation, run nemo-relay --version and confirm the version is in
the >=0.9,<0.12 range. When Relay is enabled with
FabricConfig.enable_relay(...), the Claude, Codex, and Pi adapters start and
supervise the CLI gateway automatically.
The Pi adapter also requires a user-supplied path to the
Relay Pi extension
from the Relay release that matches the CLI.
Set
harness.settings.relay_extension_path to the extension file or package
directory; relative paths resolve from environment.workspace. Relay-enabled
Pi configurations must also set runtime.artifacts so NeMo Fabric can provide
the adapter-owned Relay configuration. Refer to the
Pi adapter guide for the complete
configuration and the embedded collector streaming workflow. Do not set
launch_collector=False for Pi streaming because the Pi extension does not
attach the NeMo Fabric request IDs required by an externally managed collector.