Tool Calling
The IORails engine can sit in front of a tool-using chat model and validate tool traffic in both directions. It forwards your tool definitions to the model unchanged and applies two structural rails:
- Tool call validation checks every tool call the model emits against the tools you declared. A call must name an allowed tool and supply arguments that satisfy that tool’s JSON Schema.
- Tool result validation checks the tool results your application returns to the model. Each result must link to a tool call the model previously made, must not name a different tool than that call, and must carry well-formed content.
Both rails are local checks. They do not make extra LLM or API calls. They run on every request and fail closed: a violation, a parsing error, or a malformed payload blocks the request rather than passing it through.
The IORails engine does not execute tools. Your application or agent harness executes a tool after the model requests it, then sends the result back on the next request. These rails validate the request and response boundary; they do not call tools themselves.
LLMRails supports custom Colang flows in the rails.tool_call and
rails.tool_result sections. The built-in structural validation flows
described on this page require the opt-in, experimental IORails engine. See
Enable the IORails engine below. In
generation, these flows support only the OpenAI Chat Completions wire
format (the openai and nim engines). A check reads tool calls and
tools in that format whatever the main model’s engine, and needs no
main model.
How Function Calling Works
Your application and the model use function calling as a multi-step loop. Your application, not the model, runs the function. The flow follows the OpenAI function-calling guide. During that flow, the IORails rails validate the tool traffic at two points.
A function is a tool you define with a JSON Schema, and tool names the broader category that also covers built-in and custom tools.
When the model decides to use a tool, it returns a tool call identified by a tool_call_id, and your application answers it with a tool result that carries the same tool_call_id.
The end-to-end flow has five steps:
- Make a request to the model with tools it could call. You send the conversation along with your tool definitions.
- Receive a tool call from the model. The model responds with one or more tool calls instead of, or alongside, a text answer. IORails runs tool call validation here, before the tool calls reach your application.
- Execute code on the application side with input from the tool call. Your application runs the function and produces a result.
- Make a second request to the model with the tool output.
You resend the conversation, including the assistant turn that made the calls and one tool result for each
tool_call_id. IORails runs tool result validation here, before the model sees the results. - Receive a final response from the model (or more tool calls). The model returns a final text answer, or more tool calls that repeat the loop from step 2.
How Tool-Calling Rails Fit the Pipeline
Tool-calling rails mirror the input and output rails, with one rail per direction:
For each request, IORails runs the rails in the following order:
- Tool result validation on the incoming messages (alongside input rails).
- Input rails, then the main LLM call.
- Tool call validation on the tool calls the model returned.
- Output rails on the text response. A response that contains only tool calls and no text skips the text output rails.
When a rail blocks, IORails returns the refusal message I'm sorry, I can't respond to that. for a non-streaming request, and a guardrails_violation error payload for a streaming request.
When a rail cannot reach a verdict at all, because its provider rejected the credentials or was unreachable, the turn still stops, but the non-streaming message is I'm sorry, an internal error has occurred. instead.
That distinction lets a caller tell a provider outage, which is worth retrying, from a policy decision, which is not.
Enable the IORails Engine
The built-in validation flows described on this page require the IORails engine. Enable it in one of two ways.
Set the NEMO_GUARDRAILS_IORAILS_ENGINE environment variable so the top-level LLMRails import resolves to the IORails-backed engine.
This is the least invasive option for an existing application or the server:
Alternatively, construct the Guardrails facade directly, which uses the IORails engine by default:
If the configuration is not IORails-compatible, the Guardrails facade falls back to LLMRails and logs a warning.
Pass require_iorails=True to raise instead of falling back.
A configuration is IORails-compatible only when it uses Colang version 1.0, declares no custom llm argument, and uses only IORails-supported rails and flows.
Configure the Rails
Add the tool-calling flows to your config.yml.
When you use IORails, each tool_* section accepts only its own built-in flow name:
You can enable either rail independently.
To validate only the model’s tool calls, configure rails.tool_call and leave rails.tool_result empty.
rails.tool_output and rails.tool_input are deprecated aliases for
rails.tool_call and rails.tool_result, respectively.
The IORails flow names are fixed and direction-specific. rails.tool_call
accepts only tool call validation, and rails.tool_result accepts only
tool result validation. A misdirected, unknown, or duplicated tool flow name
makes the configuration ineligible for IORails: the Guardrails facade
silently falls back to LLMRails, and none of the IORails tool rails run. A
typo in a flow name disables the tool rails you configured, with only a log
warning to signal it. Pass require_iorails=True to the Guardrails
constructor to raise instead of falling back silently. A configuration
with per_tool rails always raises instead of falling back, and LLMRails
issues a UserWarning when it loads one, because it does not run them.
IORails honors the parallel field on the tool_call and tool_result
sections, but tool call validation and tool result validation each
name at most one flow, so parallel has no observable effect on them:
they always run one at a time. parallel matters for per-tool checks
configured under tool_call.per_tool / tool_result.per_tool, where a
request can run several checks at once; refer to
Rail Engine Support for per-tool
configuration.
Declare Tools
The rails validate against the tools you declare on the request. Tool definitions use the provider-native OpenAI Chat Completions shape, and IORails forwards them to the model unchanged.
Declare tools per request in options.llm_params:
When the model requests a tool, generate_async returns an assistant message that carries the tool calls in OpenAI shape:
After your application executes the tool, send the result back to the model.
Resend the conversation with the original user message, the assistant turn that carried the tool call, and a role: "tool" message for each tool_call_id, then call generate_async again:
The tool result validation rail runs on this request before the model sees the result, and the model then returns a final text answer or more tool calls.
You can also declare tools statically on the model in config.yml.
IORails merges these into the set of tools the rails validate against, so the rails honor a config-declared tool even when a request carries no llm_params:
Tool Call Validation
The tool call validation rail inspects every tool call in the model’s response and blocks the request when any call breaks a rule.
- Allowlist. The call must name a tool you declared.
IORails blocks a call to a tool that is not in the declared set, for example
tool call 'delete_database' is not an allowed tool. - Argument schema. The call’s arguments must validate against the tool’s declared JSON Schema (the
function.parametersblock). IORails blocks arguments that violate the schema, for examplearguments for tool 'get_weather' do not match its schema: 'required' failed at '/city'. The reason names the failing argument as the schema declares it, and the JSON Schema keyword. It never quotes an argument’s value or a key the model chose, which appears as*. - No-argument tools. A function tool that declares no parameters must receive no arguments. IORails blocks any supplied argument.
- Hosted tools. IORails allowlists a hosted or server-side tool that only
typeidentifies (for example a built-in web search) by type, and does not schema-validate its arguments because the provider owns the call shape.
If the declared schema itself is not valid JSON Schema, the rail blocks the request rather than letting an unvalidated call through.
Tool Result Validation
The tool result validation rail checks the tool results your application sends back to the model.
Because the OpenAI Chat Completions API is stateless, your application resends the conversation history, including the assistant turn that made the tool calls and the role: "tool" messages that answer them.
The rail validates each turn’s results against that same turn’s calls.
The rail blocks the request when a tool result:
- Is missing a
tool_call_id, or that id does not correspond to a tool call earlier in the conversation. - Reuses a
tool_call_idthat another result in the same turn already used. Each call must have exactly one result. - Names a different tool than the call it links to.
- Carries content that is not a string or a list of content-block objects.
A tool result doesn’t need a name.
The OpenAI tool message has none, so the rail identifies the tool from the result’s tool_call_id.
This rail validates structural well-formedness only. It confirms that the results are internally consistent with the calls in the request. It does not yet enforce a declared response schema, and it does not run a content-safety check on the tool result. Because the client resends the conversation, the linkage check verifies intra-request consistency, not server-verified provenance.
Every tool result must carry a tool_call_id that matches a tool call still
present earlier in the same request’s conversation history. If your
application truncates or prunes older turns (for example a sliding context
window), and a truncated turn held the call a later result answers, that
result becomes unresolvable. With a per-tool policy enabled
(rails.tool_result.per_tool), an unresolvable result fails closed and blocks
the request, since which policy applies can no longer be verified.
Control Rails per Request
Use options.rails to enable or disable each tool rail for a single request.
Each toggle accepts true, false, or a list of flow names, and defaults to true:
Check Tool Calls and Results Without Generation
An agent harness that calls its own model can use the same rails to validate tool traffic without IORails generating a response.
Run a check with RailType.TOOL_CALL on the conversation that ends in the model’s tool calls, and with RailType.TOOL_RESULT after appending the tool results:
A tool_call check validates the tool calls on the last assistant message against tools, which replace the tools declared on the main model.
Only a tool_call check reads tools, so passing them to any other check raises InvalidCheckRequestError, or returns HTTP 422 from the server.
Each tools entry must be a Chat Completions function tool, with a function object and a non-empty name; any other entry raises InvalidCheckRequestError rather than being dropped.
A tool check reads the messages and tools in the Chat Completions format whatever the main model’s engine, and needs no main model.
The allowlist and argument schemas are enforced for every call by tool call validation, and for the calls a per_tool rail checks by that rail; per_tool rails check only the tools they are listed under.
A tool_result check validates every tool message against the calls it answers.
Both run the global validator and any per_tool rails, and neither calls the model.
A blocked check reports each failing call or result in tool_violations, with a violation_type to switch on.
For the full behavior, the violation types, and the engine requirements, refer to Checking Messages Against Rails: Tool Rails.
To check the tool calls of a coding agent such as Claude Code or Codex from the agent’s own hooks, use the Guardrails hooks server (nemoguardrails hooks-server).
It applies a tool policy of its own to each call, then runs the called tool’s per_tool rails in this check.
The Guardrails API server (nemoguardrails server) exposes the same check through /v1/checks.
Run the server with NEMO_GUARDRAILS_IORAILS_ENGINE=1 so the configuration is served by IORails; on LLMRails, the tool rail types return HTTP 422.
Pass the tools in the request’s top-level tools field:
The call names a tool that is not declared, so the response is:
Streaming
Tool-calling rails work with stream_async.
IORails streams the text response as it arrives, accumulates the tool-call fragments, and runs tool call validation once the stream completes:
- On a clean stream, IORails emits the assembled tool calls as a final chunk after the text and output rails finish.
- When the rail blocks, IORails emits a
guardrails_violationerror payload and suppresses the tool-call chunk, so a consumer never receives a tool call after a block. - A stream that contains only tool calls and no text skips the output rails.
Limitations
- Wire format. In generation, the tool-calling rails support only the OpenAI Chat Completions shape, through the
openaiandnimengines. They do not yet support the OpenAI Responses API, Anthropic, Gemini, or Bedrock. A check takes messages and tools in the Chat Completions shape, whatever themainmodel’s engine. - Structural result validation. Tool result validation checks structure and linkage only. It does not enforce response schemas or apply content-safety checks to tool results.
- Intra-request consistency. The linkage check verifies consistency within a single request, not cross-turn provenance.
- Schema dialect. Argument validation uses JSON Schema. Tool schemas written in a different dialect are not validated against that dialect.
- Engine requirement. The built-in structural validation flows described on this page require the IORails engine and a Colang
1.0configuration.
Related Tool Features
The NVIDIA NeMo Guardrails library includes other tool-related capabilities that are distinct from the IORails tool-calling rails described here:
- Tools Integration covers LangChain tool calling and output-rail validation on the
LLMRailsengine. - Rail types describes Colang execution rails, which run actions before and after execution within the
LLMRailsevent-driven pipeline.
The rails on this page operate at the request and response boundary of the IORails engine and do not execute tools.