Skip to navigation

Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin

View as MarkdownOpen in Claude

The NVIDIA NeMo Guardrails Plugin for NeMo Relay, nvidia.nemo_guardrails, checks each tool call Relay manages before the tool runs. It sends the tool’s name and arguments to the hooks server in the NVIDIA NeMo Guardrails library at /v1/hooks/nemo-relay. The hooks server’s tool policy and per-tool rails decide whether the call runs. Configure the rails in the hooks server, which you start with nemoguardrails hooks-server.

Where the Plugin Checks Calls

The plugin registers a tool conditional-execution guardrail, which Relay runs before each tool call it manages:

Where Relay RunsCalls the Plugin Checks
The Relay CLI with Claude Code or Codex, in every modeEach PreToolUse hook. Through Relay 0.10, also each PermissionRequest hook. Refer to Check NeMo Relay Tool Calls with the Hooks Server: Check Each Call Once.
The Relay CLI with piEach tool call that Relay’s pi extension sends to Relay
A Python applicationEach tool call through nemo_relay.tools.execute, the NemoRelayMiddleware middleware for LangChain and Deep Agents, or Relay’s LangGraph tool node
OpenClawNot supported

The plugin checks tool calls only. A tool’s result reaches the model unchecked.

Prerequisites

Before you install the plugin, meet the following prerequisites:

  • NeMo Relay 0.10.1 or a later 0.10 release. Relay 0.10.0 refuses to load the plugin. In that release, an error from a guardrail lets a Claude Code tool call run.

  • A running hooks server, with a configuration for the tool names of the agent or application you check. Start the server from a clone of the repository with the configuration for your integration:

    For example, to check Claude Code, run the following command:

    nemoguardrails hooks-server --config examples/configs/agent_hooks/claude_code

    Relay forwards each tool’s name as the agent or application gives it, and the hooks server matches it exactly. For Claude Code or Codex, base the configuration on the Claude Code or Codex tool names. To write your own configuration, refer to Run the Hooks Server: Write a Hooks Configuration. If you list tools in allowed_tools, set allow_unlisted_tools: false to deny every tool you leave out: the hooks server refuses to start an allowlist that leaves it unset.

  • A clone of the NeMo Guardrails repository, rustup, a C compiler, and Python 3 to build the plugin. For the C compiler, install the Xcode Command Line Tools on macOS or build-essential on Debian and Ubuntu.

Build the Plugin Bundle

A Relay plugin bundle contains the plugin library, its manifest relay-plugin.toml, its configuration schema, and its license files. The library filename depends on your operating system:

Operating SystemLibrary Filename
Linuxlibnemo_guardrails_relay_plugin.so
macOSlibnemo_guardrails_relay_plugin.dylib
Windowsnemo_guardrails_relay_plugin.dll

Build the library and package the bundle from the repository root, passing your platform’s library to --library. On Linux, run the following commands:

cargo build --release --locked -p nemo-guardrails-relay-plugin
python3 crates/nemo-guardrails-relay-plugin/scripts/package_bundle.py \
--library target/release/libnemo_guardrails_relay_plugin.so \
--output ~/nemo-guardrails-relay-plugin

On macOS, pass --library target/release/libnemo_guardrails_relay_plugin.dylib instead. The build uses the Rust version that the repository’s rust-toolchain.toml pins.

Relay loads the plugin from the bundle directory, so keep the directory in place after you register it. The bundle is not signed, so Relay loads it only after you allow integrity-only attestation for the plugin, as the following sections show.

Update the Plugin Bundle

To update the plugin, replace the bundle at the registered path. Relay records the manifest’s path during registration and refuses to start if the path in plugins.toml differs from the registered path. Replace the entire bundle because the packaging script writes the library’s digest into the manifest. Relay refuses to load a library that does not match that digest.

Build the new library as described previously, then package it in an empty directory next to the current bundle. The packaging script rejects an output directory that contains files. Stop Relay before replacing the bundle, and ensure that the .previous backup path does not exist. On Linux, run the following commands:

python3 crates/nemo-guardrails-relay-plugin/scripts/package_bundle.py \
--library target/release/libnemo_guardrails_relay_plugin.so \
--output ~/nemo-guardrails-relay-plugin.new &&
mv ~/nemo-guardrails-relay-plugin ~/nemo-guardrails-relay-plugin.previous &&
mv ~/nemo-guardrails-relay-plugin.new ~/nemo-guardrails-relay-plugin

On macOS, use the .dylib library path instead. Validate the replacement with nemo-relay plugins validate nvidia.nemo_guardrails, then restart Relay. For a globally registered plugin, run validation with sudo. Keep the .previous bundle until you verify the replacement. If replacement or validation fails, restore the previous bundle at the registered path before restarting Relay.

Choose a Setup

The plugin works the same way in every Relay mode. How you run Relay decides who controls the plugin and its configuration:

SettingTry It OutProduction
Relay modenemo-relay claude, nemo-relay codex, nemo-relay pi, or a persistent integration from nemo-relay installThe managed Relay daemon, nemo-relay daemon
Registered withnemo-relay plugins add --usernemo-relay plugins add --global
Configuration fileYour user plugins.tomlThe system plugins.toml
Hooks serverRuns on your machine, started by youRuns where an administrator deploys it
Controlled byThe user, who can remove or reconfigure the pluginAn administrator

A Python application passes its own plugins.toml to Relay. Refer to Use the Plugin in a Python Application.

Try the Plugin Out

Register the plugin for your user, then run an agent through the Relay CLI:

  1. Start the hooks server, as described in Prerequisites.

  2. Validate the bundle, and register it for your user:

    nemo-relay plugins validate ~/nemo-guardrails-relay-plugin/relay-plugin.toml
    nemo-relay plugins add --user ~/nemo-guardrails-relay-plugin/relay-plugin.toml

    plugins add adds a [[plugins.dynamic]] entry to your user plugins.toml at $XDG_CONFIG_HOME/nemo-relay/plugins.toml. If XDG_CONFIG_HOME is not set, the file is at ~/.config/nemo-relay/plugins.toml.

  3. Open that file, allow the unsigned bundle, and add the plugin’s configuration after its entry:

    version = 1
    [plugins.policy.overrides."nvidia.nemo_guardrails"]
    attestation = "integrity_only"
    [[plugins.dynamic]]
    manifest = "/home/user/nemo-guardrails-relay-plugin/relay-plugin.toml"
    [plugins.dynamic.config]
    endpoint = "http://127.0.0.1:8100"

    Replace the example manifest value with the absolute path to your bundle’s relay-plugin.toml. Keep [plugins.dynamic.config] directly after the plugin’s [[plugins.dynamic]] entry because TOML attaches it to that entry. Without the attestation override, Relay refuses to start, with an error that the plugin requires integrity.signature under host policy.

  4. Enable the plugin, and validate its configuration:

    nemo-relay plugins enable nvidia.nemo_guardrails
    nemo-relay plugins validate nvidia.nemo_guardrails

    plugins validate checks the configuration against the plugin’s schema, including field types and the required endpoint field. It does not check the attestation override, the endpoint’s URL, or the header names. The plugin checks the URL and headers when Relay loads it. Relay refuses to start if either is invalid.

  5. Run the agent through Relay:

    nemo-relay claude

    For Codex, run nemo-relay codex against a hooks server with a configuration for the Codex tool names.

For pi, run nemo-relay pi, and also set the following:

  • NEMO_RELAY_PI_FAIL=closed in pi’s environment. By default, Relay’s standalone pi extension lets a call run when the check fails.
  • timeout_millis below the extension’s NEMO_RELAY_PI_TIMEOUT_MS limit, which defaults to 5,000 milliseconds. Otherwise, pi stops waiting before the plugin does.

Anyone who can edit your user plugins.toml can remove the plugin or point it at another hooks server. To enforce the checks for every user, refer to Run the Plugin in Production.

Verify the Plugin

With the hooks server running the Claude Code example configuration, check the plugin from Claude Code:

  1. Ask Claude to run echo 'rm -rf /tmp/guardrails-hook-check'. The example configuration blocks the command because its text matches the rm -rf pattern. Claude Code blocks the call and returns the following reason to Claude:

    guardrail rejected: NeMo Guardrails blocked tool call 'Bash': regex check tool call

    The hooks server logs the check:

    INFO:nemoguardrails.server.hooks.app:Hook check on /v1/hooks/nemo-relay: event=NemoRelayToolConditionalExecutionGuardrail tool_name='Bash' tool_use_id=None outcome=deny code=None rail='regex check tool call'
  2. Stop the hooks server and ask Claude to run any command. Claude Code blocks the call because the check fails.

  3. Start the hooks server again.

Run the Plugin in Production

In production, an administrator runs the managed Relay daemon. Its workers load the plugin for every user from the system plugins.toml of the machine they run on. Set up the daemon as described in the NeMo Relay daemon documentation, then add the plugin:

  1. Put the bundle in a directory that administrators own and every user can read, such as /opt/nemo-guardrails-relay-plugin.

  2. From an account that can write /etc/nemo-relay, register the plugin:

    sudo nemo-relay plugins add --global /opt/nemo-guardrails-relay-plugin/relay-plugin.toml
  3. Add the attestation override and plugin configuration to /etc/nemo-relay/plugins.toml, as described in Try the Plugin Out. Set endpoint to the hooks server that serves your users. Every local user can read this file, so do not put credentials in headers.

  4. Enable the plugin, and validate its configuration:

    sudo nemo-relay plugins enable nvidia.nemo_guardrails
    sudo nemo-relay plugins validate nvidia.nemo_guardrails

    Enable the plugin only after you add its override and configuration, because Relay refuses to start with the plugin enabled and either one missing. Validating a registered plugin writes its lifecycle state, so system registrations require administrator permissions.

With a remote daemon, the workers run on the client machines, not on the daemon host. Repeat these steps on every client machine with an endpoint that the client can reach. On each client, run sudo nemo-relay plugins validate nvidia.nemo_guardrails. A plugin installed only on the daemon host checks no calls from remote workers.

The daemon enforces the checks only when the following conditions hold:

  • Start the daemon with --require-worker. Without it, the daemon allows calls while no worker is ready, including during worker startup and recovery or after a daemon restart.
  • Deliver the agent’s hook settings through its managed settings. Otherwise a user can remove the hooks that send tool calls to the daemon. Claude Code has managed settings, and Codex has managed hooks in requirements.toml that users cannot disable. Refer to Codex: Deploy the Hook with Managed Settings. Relay’s managed pi extension takes effect only when pi starts with --no-extensions and the extension’s path. The user controls the command that starts pi.

The daemon’s managed pi extension blocks tool calls when a check fails and uses a fixed 30-second hook timeout. The NEMO_RELAY_PI_FAIL and NEMO_RELAY_PI_TIMEOUT_MS settings apply to the standalone pi extension, not the managed extension.

To stage a deployment, first run a daemon for your own user on one machine, as a macOS LaunchAgent or a Linux user service.

Use the Plugin in a Python Application

A Python application loads the plugin from a plugins.toml it passes to Relay. Relay’s Python package requires Python 3.11 or later. Install the same 0.10 release as the rest of your Relay setup. Include the optional langchain extra if you use Relay’s LangChain middleware:

pip install "nemo-relay[langchain]>=0.10.1,<0.11"

Write the plugins.toml file with the same entries as in Try the Plugin Out. Point manifest at the bundle you built:

version = 1
[plugins.policy.overrides."nvidia.nemo_guardrails"]
attestation = "integrity_only"
[[plugins.dynamic]]
manifest = "/home/user/nemo-guardrails-relay-plugin/relay-plugin.toml"
[plugins.dynamic.config]
endpoint = "http://127.0.0.1:8100"

Replace manifest with the absolute path to your bundle’s relay-plugin.toml. A relative manifest path resolves from the directory that contains plugins.toml, for example when the bundle ships with your application.

Activate Relay’s plugins with the file, check that the plugin loaded, and run the agent while the plugins are active:

import asyncio
from pathlib import Path
from nemo_relay import plugin
async def main() -> None:
async with plugin.activate({}, Path(__file__).with_name("plugins.toml")) as activation:
report = activation.report
loaded = {entry["plugin_id"] for entry in report["dynamic_plugins"] if entry["selected"]}
if "nvidia.nemo_guardrails" not in loaded:
raise RuntimeError(f"NeMo Guardrails plugin not loaded: {report['config']['diagnostics']}")
... # Run your agent here.
asyncio.run(main())

If the plugins.toml file does not exist, Relay adds a plugin.configuration_file_missing warning and continues activation. Without another configuration that loads the plugin, tool calls run unchecked. The example resolves the path from the script’s location and stops if the plugin is not loaded.

Relay also reads the system plugins.toml, which is /etc/nemo-relay/plugins.toml on Linux and macOS. Its settings take precedence over the file you pass.

For a LangChain or Deep Agents agent, add Relay’s middleware, which runs each tool call through Relay:

from langchain.agents import create_agent
from nemo_relay.integrations.langchain.middleware import NemoRelayMiddleware
agent = create_agent(model=model, tools=tools, middleware=[NemoRelayMiddleware()])

When the hooks server denies a call, invoke and ainvoke raise a RuntimeError, and the tool does not run:

RuntimeError: guardrail rejected: NeMo Guardrails blocked tool call 'search': regex check tool call

A failed check raises a RuntimeError whose message starts with internal error: NeMo Guardrails check failed for tool call. Neither error reaches the model. Handle both in your application’s error handler.

A Python application has no hook time limit, so the default timeout_millis leaves room for a rail that calls a model.

Configure the Plugin

The plugin reads its configuration from the [plugins.dynamic.config] table of its entry:

FieldDefaultDescription
endpointNoneRequired. The base URL of the hooks server, using http or https, such as http://127.0.0.1:8100. The plugin sends each check to /v1/hooks/nemo-relay under it, keeping any path prefix. The URL must not contain a username or password. Send credentials in headers.
headers{}Header names and string values to send with every check. Use headers to authenticate with a proxy or gateway, such as through an authorization header. The hooks server itself has no authentication. The plugin rejects headers that its HTTP client controls, including content-type, content-length, and host. For error-message handling, refer to What Happens to a Call.
timeout_millis25000A positive integer that sets the time limit for one check, in milliseconds. Keep it below the integration’s hook timeout. Claude Code, Codex, and managed pi use 30 seconds. Standalone pi defaults to five seconds.
priority0The plugin’s place among Relay’s tool guardrails. Lower values run earlier.
executorNoneOptional Relay software development kit (SDK) executor settings. worker_threads must be a positive integer.

The following example sends an authorization header to a proxy in front of the hooks server:

[plugins.dynamic.config]
endpoint = "https://guardrails.example.com"
timeout_millis = 10000
[plugins.dynamic.config.headers]
authorization = "Bearer <token>"

Relay refuses to start when the configuration is not valid.

The plugin uses the proxy settings of the Relay process that loads it, from HTTP_PROXY, HTTPS_PROXY, ALL_PROXY, and NO_PROXY, even for a hooks server on 127.0.0.1. If you set a proxy, add the hooks server’s host to NO_PROXY, such as NO_PROXY=127.0.0.1,localhost. Otherwise, checks go through the proxy, which receives the tool’s arguments and configured headers. An http endpoint sends these values in plain text.

What Happens to a Call

The plugin turns the hooks server’s answer into the guardrail’s result, and Relay acts on it:

Hooks Server AnswerPlugin ResultRelay CLI with Claude Code or CodexStandalone pi ExtensionPython Application
AllowLets the call runThe tool runs. Through Relay 0.10, Relay also approves Claude Code’s permission prompt for the call, so Claude Code does not ask the user. Codex applies its own approval settings.The tool runsThe tool runs
DenyA rejection: NeMo Guardrails blocked tool call '<tool>': <reason>Relay’s hook exits 2, and the agent blocks the callpi blocks the callRuntimeError: guardrail rejected: ...
A status other than 200, an invalid decision body, a timeout, or a failed connectionAn error: NeMo Guardrails check failed for tool call '<tool>': <detail>Relay’s hook exits 2, and the agent blocks the callThe tool runs, unless NEMO_RELAY_PI_FAIL=closedRuntimeError: internal error: ...

The managed pi extension blocks both denied calls and failed checks.

The reason for a deny comes from the hooks server: the tool policy’s reason, or the name of the per-tool rail that blocked the call. For every deny reason and error, refer to Hooks Server Results and Errors.

The detail of a failed check gives the HTTP status, the hooks server’s error code and message, or says that the request timed out or could not be sent. The plugin adds no header value or tool argument to it, but it repeats the server’s error message, up to 512 characters. A proxy or gateway in front of the hooks server could echo a header or an argument in that message. The plugin does not retry failed checks, including a 503 response with Retry-After.

If the hooks server is stopped, restarting, or slower than timeout_millis, every check fails. The integration blocks these tool calls unless you use the standalone pi extension without NEMO_RELAY_PI_FAIL=closed. The same holds for the calls a per-tool rail checks while the model it calls cannot be reached.

Find the Reason for a Failed Check

Through the Relay CLI’s gateway, the agent receives the plugin’s error inside Relay’s message:

nemo-relay hook forward failed: hook gateway returned HTTP 500 Internal Server Error: NeMo Relay runtime error: internal error: NeMo Guardrails check failed for tool call 'Bash': error sending request for url (http://127.0.0.1:8100/v1/hooks/nemo-relay): client error (Connect): tcp connect error: Connection refused (os error 61)

Through the managed daemon, the agent can receive only Relay’s own message, such as daemon hook forward failed with HTTP 500.

The plugin also logs each failed check as a warning in Relay’s log. The warning uses the target nemo_relay.plugin.native.nvidia.nemo_guardrails, the event nemo_guardrails_check_failed, and the fields tool_name, status, and error_code:

2026-10-28T18:09:36.742Z WARN root=01a183b3 target=nemo_relay.plugin.native.nvidia.nemo_guardrails event=nemo_guardrails_check_failed error_code=None status=None tool_name=Bash NeMo Guardrails check failed for tool call 'Bash': error sending request for url (http://127.0.0.1:8100/v1/hooks/nemo-relay): client error (Connect): tcp connect error: Connection refused (os error 61)

Relay logs only errors by default. Set its log level to warn or a more detailed level to retain these warnings, for example with nemo-relay --log-level warn. The hooks server logs each check that ends in an error at the WARNING level, with its error code.

Limits

Plan your deployment around the following limits:

  • No result checks. The hooks server checks tool calls only, so a tool’s output reaches the model unchecked.
  • One configuration for each Relay process. Relay loads the plugin once in each process, with one endpoint, and a hooks server serves one configuration. To apply another policy, run another hooks server, and point another Relay setup at it.
  • No authentication on the hooks server. Anyone who can reach the hooks server can send it calls and learn what the policy allows. Refer to Run the Hooks Server: Expose the Server.