Skip to navigation

The IORails engine can sit in front of a tool-using chat model and validate tool traffic in both directions. It forwards your tool definitions to the model unchanged and applies two structural rails:

  • Tool call validation checks every tool call the model emits against the tools you declared. A call must name an allowed tool and supply arguments that satisfy that tool’s JSON Schema.
  • Tool result validation checks the tool results your application returns to the model. Each result must link to a tool call the model previously made, must not name a different tool than that call, and must carry well-formed content.

Both rails are local checks. They do not make extra LLM or API calls. They run on every request and fail closed: a violation, a parsing error, or a malformed payload blocks the request rather than passing it through.

The IORails engine does not execute tools. Your application or agent harness executes a tool after the model requests it, then sends the result back on the next request. These rails validate the request and response boundary; they do not call tools themselves.

LLMRails supports custom Colang flows in the rails.tool_call and rails.tool_result sections. The built-in structural validation flows described on this page require the opt-in, experimental IORails engine. See Enable the IORails engine below. In generation, these flows support only the OpenAI Chat Completions wire format (the openai and nim engines). A check reads tool calls and tools in that format whatever the main model’s engine, and needs no main model.

How Function Calling Works

Your application and the model use function calling as a multi-step loop. Your application, not the model, runs the function. The flow follows the OpenAI function-calling guide. During that flow, the IORails rails validate the tool traffic at two points.

A function is a tool you define with a JSON Schema, and tool names the broader category that also covers built-in and custom tools. When the model decides to use a tool, it returns a tool call identified by a tool_call_id, and your application answers it with a tool result that carries the same tool_call_id.

The end-to-end flow has five steps:

  1. Make a request to the model with tools it could call. You send the conversation along with your tool definitions.
  2. Receive a tool call from the model. The model responds with one or more tool calls instead of, or alongside, a text answer. IORails runs tool call validation here, before the tool calls reach your application.
  3. Execute code on the application side with input from the tool call. Your application runs the function and produces a result.
  4. Make a second request to the model with the tool output. You resend the conversation, including the assistant turn that made the calls and one tool result for each tool_call_id. IORails runs tool result validation here, before the model sees the results.
  5. Receive a final response from the model (or more tool calls). The model returns a final text answer, or more tool calls that repeat the loop from step 2.
1. Request with tool definitions Forward request 2. Tool calls Validated tool calls 3. Execute the function 4. Request with tool results Forward request 5. Final response or more tool calls Final response tool call validation tool result validation Application IORails Model

How Tool-Calling Rails Fit the Pipeline

Tool-calling rails mirror the input and output rails, with one rail per direction:

RailConfig sectionFlow nameDirectionWhat it checks
Tool call validationrails.tool_calltool call validationModel outputTool calls the model emits: allowed name plus schema-valid arguments.
Tool result validationrails.tool_resulttool result validationRequest inputTool results sent back to the model: linkage to a prior call, name consistency, well-formed content.

For each request, IORails runs the rails in the following order:

  1. Tool result validation on the incoming messages (alongside input rails).
  2. Input rails, then the main LLM call.
  3. Tool call validation on the tool calls the model returned.
  4. Output rails on the text response. A response that contains only tool calls and no text skips the text output rails.

When a rail blocks, IORails returns the refusal message I'm sorry, I can't respond to that. for a non-streaming request, and a guardrails_violation error payload for a streaming request. When a rail cannot reach a verdict at all, because its provider rejected the credentials or was unreachable, the turn still stops, but the non-streaming message is I'm sorry, an internal error has occurred. instead. That distinction lets a caller tell a provider outage, which is worth retrying, from a policy decision, which is not.

Enable the IORails Engine

The built-in validation flows described on this page require the IORails engine. Enable it in one of two ways.

Set the NEMO_GUARDRAILS_IORAILS_ENGINE environment variable so the top-level LLMRails import resolves to the IORails-backed engine. This is the least invasive option for an existing application or the server:

$ export NEMO_GUARDRAILS_IORAILS_ENGINE=1
from nemoguardrails import LLMRails, RailsConfig
config = RailsConfig.from_path("./config")
rails = LLMRails(config)

Alternatively, construct the Guardrails facade directly, which uses the IORails engine by default:

from nemoguardrails import Guardrails, RailsConfig
config = RailsConfig.from_path("./config")
rails = Guardrails(config)

If the configuration is not IORails-compatible, the Guardrails facade falls back to LLMRails and logs a warning. Pass require_iorails=True to raise instead of falling back. A configuration is IORails-compatible only when it uses Colang version 1.0, declares no custom llm argument, and uses only IORails-supported rails and flows.

Configure the Rails

Add the tool-calling flows to your config.yml. When you use IORails, each tool_* section accepts only its own built-in flow name:

models:
- type: main
engine: nim
model: meta/llama-3.3-70b-instruct
rails:
tool_call:
flows:
- tool call validation
tool_result:
flows:
- tool result validation

You can enable either rail independently. To validate only the model’s tool calls, configure rails.tool_call and leave rails.tool_result empty.

rails.tool_output and rails.tool_input are deprecated aliases for rails.tool_call and rails.tool_result, respectively.

The IORails flow names are fixed and direction-specific. rails.tool_call accepts only tool call validation, and rails.tool_result accepts only tool result validation. A misdirected, unknown, or duplicated tool flow name makes the configuration ineligible for IORails: the Guardrails facade silently falls back to LLMRails, and none of the IORails tool rails run. A typo in a flow name disables the tool rails you configured, with only a log warning to signal it. Pass require_iorails=True to the Guardrails constructor to raise instead of falling back silently. A configuration with per_tool rails always raises instead of falling back, and LLMRails issues a UserWarning when it loads one, because it does not run them.

IORails honors the parallel field on the tool_call and tool_result sections, but tool call validation and tool result validation each name at most one flow, so parallel has no observable effect on them: they always run one at a time. parallel matters for per-tool checks configured under tool_call.per_tool / tool_result.per_tool, where a request can run several checks at once; refer to Rail Engine Support for per-tool configuration.

Declare Tools

The rails validate against the tools you declare on the request. Tool definitions use the provider-native OpenAI Chat Completions shape, and IORails forwards them to the model unchanged.

Declare tools per request in options.llm_params:

tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the weather for a city.",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
},
},
]
response = await rails.generate_async(
messages=[{"role": "user", "content": "What's the weather in Paris?"}],
options={"llm_params": {"tools": tools, "tool_choice": "auto"}},
)

When the model requests a tool, generate_async returns an assistant message that carries the tool calls in OpenAI shape:

{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "call_1",
"type": "function",
"function": {"name": "get_weather", "arguments": "{\"city\": \"Paris\"}"},
}
],
}

After your application executes the tool, send the result back to the model. Resend the conversation with the original user message, the assistant turn that carried the tool call, and a role: "tool" message for each tool_call_id, then call generate_async again:

messages = [
{"role": "user", "content": "What's the weather in Paris?"},
{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "call_1",
"type": "function",
"function": {"name": "get_weather", "arguments": "{\"city\": \"Paris\"}"},
}
],
},
{
"role": "tool",
"tool_call_id": "call_1",
"name": "get_weather",
"content": "{\"temperature_c\": 18, \"condition\": \"cloudy\"}",
},
]
response = await rails.generate_async(
messages=messages,
options={"llm_params": {"tools": tools}},
)

The tool result validation rail runs on this request before the model sees the result, and the model then returns a final text answer or more tool calls.

You can also declare tools statically on the model in config.yml. IORails merges these into the set of tools the rails validate against, so the rails honor a config-declared tool even when a request carries no llm_params:

models:
- type: main
engine: nim
model: meta/llama-3.3-70b-instruct
parameters:
tools:
- type: function
function:
name: get_weather
description: Get the weather for a city.
parameters:
type: object
properties:
city:
type: string
required:
- city

Tool Call Validation

The tool call validation rail inspects every tool call in the model’s response and blocks the request when any call breaks a rule.

  • Allowlist. The call must name a tool you declared. IORails blocks a call to a tool that is not in the declared set, for example tool call 'delete_database' is not an allowed tool.
  • Argument schema. The call’s arguments must validate against the tool’s declared JSON Schema (the function.parameters block). IORails blocks arguments that violate the schema, for example arguments for tool 'get_weather' do not match its schema: 'required' failed at '/city'. The reason names the failing argument as the schema declares it, and the JSON Schema keyword. It never quotes an argument’s value or a key the model chose, which appears as *.
  • No-argument tools. A function tool that declares no parameters must receive no arguments. IORails blocks any supplied argument.
  • Hosted tools. IORails allowlists a hosted or server-side tool that only type identifies (for example a built-in web search) by type, and does not schema-validate its arguments because the provider owns the call shape.

If the declared schema itself is not valid JSON Schema, the rail blocks the request rather than letting an unvalidated call through.

Tool Result Validation

The tool result validation rail checks the tool results your application sends back to the model. Because the OpenAI Chat Completions API is stateless, your application resends the conversation history, including the assistant turn that made the tool calls and the role: "tool" messages that answer them. The rail validates each turn’s results against that same turn’s calls.

The rail blocks the request when a tool result:

  • Is missing a tool_call_id, or that id does not correspond to a tool call earlier in the conversation.
  • Reuses a tool_call_id that another result in the same turn already used. Each call must have exactly one result.
  • Names a different tool than the call it links to.
  • Carries content that is not a string or a list of content-block objects.

A tool result doesn’t need a name. The OpenAI tool message has none, so the rail identifies the tool from the result’s tool_call_id.

This rail validates structural well-formedness only. It confirms that the results are internally consistent with the calls in the request. It does not yet enforce a declared response schema, and it does not run a content-safety check on the tool result. Because the client resends the conversation, the linkage check verifies intra-request consistency, not server-verified provenance.

Every tool result must carry a tool_call_id that matches a tool call still present earlier in the same request’s conversation history. If your application truncates or prunes older turns (for example a sliding context window), and a truncated turn held the call a later result answers, that result becomes unresolvable. With a per-tool policy enabled (rails.tool_result.per_tool), an unresolvable result fails closed and blocks the request, since which policy applies can no longer be verified.

Control Rails per Request

Use options.rails to enable or disable each tool rail for a single request. Each toggle accepts true, false, or a list of flow names, and defaults to true:

response = await rails.generate_async(
messages=messages,
options={
"llm_params": {"tools": tools},
"rails": {"tool_call": True, "tool_result": False},
},
)

Check Tool Calls and Results Without Generation

An agent harness that calls its own model can use the same rails to validate tool traffic without IORails generating a response. Run a check with RailType.TOOL_CALL on the conversation that ends in the model’s tool calls, and with RailType.TOOL_RESULT after appending the tool results:

from nemoguardrails import Guardrails, RailsConfig
from nemoguardrails.rails.llm.options import RailStatus, RailType
rails = Guardrails(RailsConfig.from_path("./config"), require_iorails=True)
call_check = await rails.check_async(messages, rail_types=[RailType.TOOL_CALL], tools=tools)
if call_check.status == RailStatus.BLOCKED:
... # Drop or re-request the calls named in call_check.tool_violations.
messages.append({"role": "tool", "tool_call_id": "call_1", "content": "18C"})
result_check = await rails.check_async(messages, rail_types=[RailType.TOOL_RESULT])

A tool_call check validates the tool calls on the last assistant message against tools, which replace the tools declared on the main model. Only a tool_call check reads tools, so passing them to any other check raises InvalidCheckRequestError, or returns HTTP 422 from the server. Each tools entry must be a Chat Completions function tool, with a function object and a non-empty name; any other entry raises InvalidCheckRequestError rather than being dropped. A tool check reads the messages and tools in the Chat Completions format whatever the main model’s engine, and needs no main model. The allowlist and argument schemas are enforced for every call by tool call validation, and for the calls a per_tool rail checks by that rail; per_tool rails check only the tools they are listed under. A tool_result check validates every tool message against the calls it answers. Both run the global validator and any per_tool rails, and neither calls the model. A blocked check reports each failing call or result in tool_violations, with a violation_type to switch on. For the full behavior, the violation types, and the engine requirements, refer to Checking Messages Against Rails: Tool Rails.

To check the tool calls of a coding agent such as Claude Code or Codex from the agent’s own hooks, use the Guardrails hooks server (nemoguardrails hooks-server). It applies a tool policy of its own to each call, then runs the called tool’s per_tool rails in this check.

The Guardrails API server (nemoguardrails server) exposes the same check through /v1/checks. Run the server with NEMO_GUARDRAILS_IORAILS_ENGINE=1 so the configuration is served by IORails; on LLMRails, the tool rail types return HTTP 422. Pass the tools in the request’s top-level tools field:

{
"model": "meta/llama-3.3-70b-instruct",
"messages": [
{"role": "user", "content": "What's the weather in Paris?"},
{"role": "assistant", "content": null, "tool_calls": [
{"id": "call_1", "type": "function", "function": {"name": "delete_files", "arguments": "{}"}}
]}
],
"tools": [{"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}],
"guardrails": {"config_id": "tools", "rail_types": ["tool_call"]}
}

The call names a tool that is not declared, so the response is:

{
"status": "blocked",
"content": "I'm sorry, I can't respond to that.",
"rail": "tool call validation",
"reason": "tool call 'delete_files' is not an allowed tool",
"tool_violations": [
{"kind": "tool_call", "violation_type": "tool_not_allowed", "reason": "tool call 'delete_files' is not an allowed tool", "tool_call_id": "call_1", "tool_name": "delete_files", "index": 0}
]
}

Streaming

Tool-calling rails work with stream_async. IORails streams the text response as it arrives, accumulates the tool-call fragments, and runs tool call validation once the stream completes:

  • On a clean stream, IORails emits the assembled tool calls as a final chunk after the text and output rails finish.
  • When the rail blocks, IORails emits a guardrails_violation error payload and suppresses the tool-call chunk, so a consumer never receives a tool call after a block.
  • A stream that contains only tool calls and no text skips the output rails.

Limitations

  • Wire format. In generation, the tool-calling rails support only the OpenAI Chat Completions shape, through the openai and nim engines. They do not yet support the OpenAI Responses API, Anthropic, Gemini, or Bedrock. A check takes messages and tools in the Chat Completions shape, whatever the main model’s engine.
  • Structural result validation. Tool result validation checks structure and linkage only. It does not enforce response schemas or apply content-safety checks to tool results.
  • Intra-request consistency. The linkage check verifies consistency within a single request, not cross-turn provenance.
  • Schema dialect. Argument validation uses JSON Schema. Tool schemas written in a different dialect are not validated against that dialect.
  • Engine requirement. The built-in structural validation flows described on this page require the IORails engine and a Colang 1.0 configuration.

The NVIDIA NeMo Guardrails library includes other tool-related capabilities that are distinct from the IORails tool-calling rails described here:

  • Tools Integration covers LangChain tool calling and output-rail validation on the LLMRails engine.
  • Rail types describes Colang execution rails, which run actions before and after execution within the LLMRails event-driven pipeline.

The rails on this page operate at the request and response boundary of the IORails engine and do not execute tools.