Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin
The NVIDIA NeMo Guardrails Plugin for NeMo Relay, nvidia.nemo_guardrails, checks each tool call Relay manages before the tool runs.
It sends the tool’s name and arguments to the hooks server in the NVIDIA NeMo Guardrails library at /v1/hooks/nemo-relay.
The hooks server’s tool policy and per-tool rails decide whether the call runs.
Configure the rails in the hooks server, which you start with nemoguardrails hooks-server.
Where the Plugin Checks Calls
The plugin registers a tool conditional-execution guardrail, which Relay runs before each tool call it manages:
The plugin checks tool calls only. A tool’s result reaches the model unchecked.
Prerequisites
Before you install the plugin, meet the following prerequisites:
-
NeMo Relay 0.10.1 or a later 0.10 release. Relay 0.10.0 refuses to load the plugin. In that release, an error from a guardrail lets a Claude Code tool call run.
-
A running hooks server, with a configuration for the tool names of the agent or application you check. Start the server from a clone of the repository with the configuration for your integration:
- Claude Code, through
nemo-relay claude: the Claude Code example configuration. - Codex, through
nemo-relay codex: the Codex example configuration. - A Python application: the NeMo Relay example configuration, which allows only a
searchtool. The Python examples on this page use it. - pi: Write a configuration for pi’s tool names. The repository has no pi example configuration.
For example, to check Claude Code, run the following command:
Relay forwards each tool’s name as the agent or application gives it, and the hooks server matches it exactly. For Claude Code or Codex, base the configuration on the Claude Code or Codex tool names. To write your own configuration, refer to Run the Hooks Server: Write a Hooks Configuration. If you list tools in
allowed_tools, setallow_unlisted_tools: falseto deny every tool you leave out: the hooks server refuses to start an allowlist that leaves it unset. - Claude Code, through
-
A clone of the NeMo Guardrails repository, rustup, a C compiler, and Python 3 to build the plugin. For the C compiler, install the Xcode Command Line Tools on macOS or
build-essentialon Debian and Ubuntu.
Build the Plugin Bundle
A Relay plugin bundle contains the plugin library, its manifest relay-plugin.toml, its configuration schema, and its license files.
The library filename depends on your operating system:
Build the library and package the bundle from the repository root, passing your platform’s library to --library.
On Linux, run the following commands:
On macOS, pass --library target/release/libnemo_guardrails_relay_plugin.dylib instead.
The build uses the Rust version that the repository’s rust-toolchain.toml pins.
Relay loads the plugin from the bundle directory, so keep the directory in place after you register it. The bundle is not signed, so Relay loads it only after you allow integrity-only attestation for the plugin, as the following sections show.
Update the Plugin Bundle
To update the plugin, replace the bundle at the registered path.
Relay records the manifest’s path during registration and refuses to start if the path in plugins.toml differs from the registered path.
Replace the entire bundle because the packaging script writes the library’s digest into the manifest.
Relay refuses to load a library that does not match that digest.
Build the new library as described previously, then package it in an empty directory next to the current bundle.
The packaging script rejects an output directory that contains files.
Stop Relay before replacing the bundle, and ensure that the .previous backup path does not exist.
On Linux, run the following commands:
On macOS, use the .dylib library path instead.
Validate the replacement with nemo-relay plugins validate nvidia.nemo_guardrails, then restart Relay.
For a globally registered plugin, run validation with sudo.
Keep the .previous bundle until you verify the replacement.
If replacement or validation fails, restore the previous bundle at the registered path before restarting Relay.
Choose a Setup
The plugin works the same way in every Relay mode. How you run Relay decides who controls the plugin and its configuration:
A Python application passes its own plugins.toml to Relay.
Refer to Use the Plugin in a Python Application.
Try the Plugin Out
Register the plugin for your user, then run an agent through the Relay CLI:
-
Start the hooks server, as described in Prerequisites.
-
Validate the bundle, and register it for your user:
plugins addadds a[[plugins.dynamic]]entry to your userplugins.tomlat$XDG_CONFIG_HOME/nemo-relay/plugins.toml. IfXDG_CONFIG_HOMEis not set, the file is at~/.config/nemo-relay/plugins.toml. -
Open that file, allow the unsigned bundle, and add the plugin’s configuration after its entry:
Replace the example
manifestvalue with the absolute path to your bundle’srelay-plugin.toml. Keep[plugins.dynamic.config]directly after the plugin’s[[plugins.dynamic]]entry because TOML attaches it to that entry. Without theattestationoverride, Relay refuses to start, with an error that the plugin requiresintegrity.signatureunder host policy. -
Enable the plugin, and validate its configuration:
plugins validatechecks the configuration against the plugin’s schema, including field types and the requiredendpointfield. It does not check theattestationoverride, the endpoint’s URL, or the header names. The plugin checks the URL and headers when Relay loads it. Relay refuses to start if either is invalid. -
Run the agent through Relay:
For Codex, run
nemo-relay codexagainst a hooks server with a configuration for the Codex tool names.
For pi, run nemo-relay pi, and also set the following:
NEMO_RELAY_PI_FAIL=closedin pi’s environment. By default, Relay’s standalone pi extension lets a call run when the check fails.timeout_millisbelow the extension’sNEMO_RELAY_PI_TIMEOUT_MSlimit, which defaults to 5,000 milliseconds. Otherwise, pi stops waiting before the plugin does.
Anyone who can edit your user plugins.toml can remove the plugin or point it at another hooks server.
To enforce the checks for every user, refer to Run the Plugin in Production.
Verify the Plugin
With the hooks server running the Claude Code example configuration, check the plugin from Claude Code:
-
Ask Claude to run
echo 'rm -rf /tmp/guardrails-hook-check'. The example configuration blocks the command because its text matches therm -rfpattern. Claude Code blocks the call and returns the following reason to Claude:The hooks server logs the check:
-
Stop the hooks server and ask Claude to run any command. Claude Code blocks the call because the check fails.
-
Start the hooks server again.
Run the Plugin in Production
In production, an administrator runs the managed Relay daemon.
Its workers load the plugin for every user from the system plugins.toml of the machine they run on.
Set up the daemon as described in the NeMo Relay daemon documentation, then add the plugin:
-
Put the bundle in a directory that administrators own and every user can read, such as
/opt/nemo-guardrails-relay-plugin. -
From an account that can write
/etc/nemo-relay, register the plugin: -
Add the
attestationoverride and plugin configuration to/etc/nemo-relay/plugins.toml, as described in Try the Plugin Out. Setendpointto the hooks server that serves your users. Every local user can read this file, so do not put credentials inheaders. -
Enable the plugin, and validate its configuration:
Enable the plugin only after you add its override and configuration, because Relay refuses to start with the plugin enabled and either one missing. Validating a registered plugin writes its lifecycle state, so system registrations require administrator permissions.
With a remote daemon, the workers run on the client machines, not on the daemon host.
Repeat these steps on every client machine with an endpoint that the client can reach.
On each client, run sudo nemo-relay plugins validate nvidia.nemo_guardrails.
A plugin installed only on the daemon host checks no calls from remote workers.
The daemon enforces the checks only when the following conditions hold:
- Start the daemon with
--require-worker. Without it, the daemon allows calls while no worker is ready, including during worker startup and recovery or after a daemon restart. - Deliver the agent’s hook settings through its managed settings.
Otherwise a user can remove the hooks that send tool calls to the daemon.
Claude Code has managed settings, and Codex has managed hooks in
requirements.tomlthat users cannot disable. Refer to Codex: Deploy the Hook with Managed Settings. Relay’s managed pi extension takes effect only when pi starts with--no-extensionsand the extension’s path. The user controls the command that starts pi.
The daemon’s managed pi extension blocks tool calls when a check fails and uses a fixed 30-second hook timeout.
The NEMO_RELAY_PI_FAIL and NEMO_RELAY_PI_TIMEOUT_MS settings apply to the standalone pi extension, not the managed extension.
To stage a deployment, first run a daemon for your own user on one machine, as a macOS LaunchAgent or a Linux user service.
Use the Plugin in a Python Application
A Python application loads the plugin from a plugins.toml it passes to Relay.
Relay’s Python package requires Python 3.11 or later.
Install the same 0.10 release as the rest of your Relay setup.
Include the optional langchain extra if you use Relay’s LangChain middleware:
Write the plugins.toml file with the same entries as in Try the Plugin Out.
Point manifest at the bundle you built:
Replace manifest with the absolute path to your bundle’s relay-plugin.toml.
A relative manifest path resolves from the directory that contains plugins.toml, for example when the bundle ships with your application.
Activate Relay’s plugins with the file, check that the plugin loaded, and run the agent while the plugins are active:
If the plugins.toml file does not exist, Relay adds a plugin.configuration_file_missing warning and continues activation.
Without another configuration that loads the plugin, tool calls run unchecked.
The example resolves the path from the script’s location and stops if the plugin is not loaded.
Relay also reads the system plugins.toml, which is /etc/nemo-relay/plugins.toml on Linux and macOS.
Its settings take precedence over the file you pass.
For a LangChain or Deep Agents agent, add Relay’s middleware, which runs each tool call through Relay:
When the hooks server denies a call, invoke and ainvoke raise a RuntimeError, and the tool does not run:
A failed check raises a RuntimeError whose message starts with internal error: NeMo Guardrails check failed for tool call.
Neither error reaches the model.
Handle both in your application’s error handler.
A Python application has no hook time limit, so the default timeout_millis leaves room for a rail that calls a model.
Configure the Plugin
The plugin reads its configuration from the [plugins.dynamic.config] table of its entry:
The following example sends an authorization header to a proxy in front of the hooks server:
Relay refuses to start when the configuration is not valid.
The plugin uses the proxy settings of the Relay process that loads it, from HTTP_PROXY, HTTPS_PROXY, ALL_PROXY, and NO_PROXY, even for a hooks server on 127.0.0.1.
If you set a proxy, add the hooks server’s host to NO_PROXY, such as NO_PROXY=127.0.0.1,localhost.
Otherwise, checks go through the proxy, which receives the tool’s arguments and configured headers.
An http endpoint sends these values in plain text.
What Happens to a Call
The plugin turns the hooks server’s answer into the guardrail’s result, and Relay acts on it:
The managed pi extension blocks both denied calls and failed checks.
The reason for a deny comes from the hooks server: the tool policy’s reason, or the name of the per-tool rail that blocked the call. For every deny reason and error, refer to Hooks Server Results and Errors.
The detail of a failed check gives the HTTP status, the hooks server’s error code and message, or says that the request timed out or could not be sent.
The plugin adds no header value or tool argument to it, but it repeats the server’s error message, up to 512 characters.
A proxy or gateway in front of the hooks server could echo a header or an argument in that message.
The plugin does not retry failed checks, including a 503 response with Retry-After.
If the hooks server is stopped, restarting, or slower than timeout_millis, every check fails.
The integration blocks these tool calls unless you use the standalone pi extension without NEMO_RELAY_PI_FAIL=closed.
The same holds for the calls a per-tool rail checks while the model it calls cannot be reached.
Find the Reason for a Failed Check
Through the Relay CLI’s gateway, the agent receives the plugin’s error inside Relay’s message:
Through the managed daemon, the agent can receive only Relay’s own message, such as daemon hook forward failed with HTTP 500.
The plugin also logs each failed check as a warning in Relay’s log.
The warning uses the target nemo_relay.plugin.native.nvidia.nemo_guardrails, the event nemo_guardrails_check_failed, and the fields tool_name, status, and error_code:
Relay logs only errors by default.
Set its log level to warn or a more detailed level to retain these warnings, for example with nemo-relay --log-level warn.
The hooks server logs each check that ends in an error at the WARNING level, with its error code.
Limits
Plan your deployment around the following limits:
- No result checks. The hooks server checks tool calls only, so a tool’s output reaches the model unchecked.
- One configuration for each Relay process.
Relay loads the plugin once in each process, with one
endpoint, and a hooks server serves one configuration. To apply another policy, run another hooks server, and point another Relay setup at it. - No authentication on the hooks server. Anyone who can reach the hooks server can send it calls and learn what the policy allows. Refer to Run the Hooks Server: Expose the Server.
Related Topics
- Check NeMo Relay Tool Calls with the Hooks Server
- Run the Hooks Server
- Hooks Server Results and Errors
- Configure Discoverable Plugins in the NeMo Relay documentation
- The plugin’s README, for building and testing it