> This page is for version v0.25.0.
> For other versions, use one of these documentation indexes:
> - Latest (default): https://docs.nvidia.com/nemo/guardrails/latest/llms.txt
> - v0.25.0: https://docs.nvidia.com/nemo/guardrails/v0.25.0/llms.txt
> - v0.24.1: https://docs.nvidia.com/nemo/guardrails/v0.24.1/llms.txt
> - v0.24.0: https://docs.nvidia.com/nemo/guardrails/v0.24.0/llms.txt
> - v0.23.0: https://docs.nvidia.com/nemo/guardrails/v0.23.0/llms.txt
> - v0.22.0: https://docs.nvidia.com/nemo/guardrails/v0.22.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/guardrails/llms.txt.

# Tool Calling

> Validate model-emitted tool calls and application-returned tool results with the IORails tool-calling rails.

The IORails engine can sit in front of a tool-using chat model and validate tool traffic in both directions.
It forwards your tool definitions to the model unchanged and applies two structural rails:

* **Tool call validation** checks every tool call the model emits against the tools you declared.
  A call must name an allowed tool and supply arguments that satisfy that tool's JSON Schema.
* **Tool result validation** checks the tool results your application returns to the model.
  Each result must link to a tool call the model previously made, must not name a different tool than that call, and must carry well-formed content.

Both rails are local checks.
They do not make extra LLM or API calls.
They run on every request and fail closed: a violation, a parsing error, or a malformed payload blocks the request rather than passing it through.

> **Note**
>
> The IORails engine does not execute tools. Your application or agent harness
> executes a tool after the model requests it, then sends the result back on
> the next request. These rails validate the request and response boundary;
> they do not call tools themselves.

> **Warning**
>
> `LLMRails` supports custom Colang flows in the `rails.tool_call` and
> `rails.tool_result` sections. The built-in structural validation flows
> described on this page require the opt-in, experimental IORails engine. See
> [Enable the IORails engine](#enable-the-iorails-engine) below. In
> generation, these flows support only the OpenAI Chat Completions wire
> format (the `openai` and `nim` engines). A check reads tool calls and
> tools in that format whatever the `main` model's engine, and needs no
> `main` model.

## How Function Calling Works

Your application and the model use function calling as a multi-step loop.
Your application, not the model, runs the function.
The flow follows the [OpenAI function-calling guide](https://developers.openai.com/api/docs/guides/function-calling).
During that flow, the IORails rails validate the tool traffic at two points.

A *function* is a tool you define with a JSON Schema, and *tool* names the broader category that also covers built-in and custom tools.
When the model decides to use a tool, it returns a *tool call* identified by a `tool_call_id`, and your application answers it with a *tool result* that carries the same `tool_call_id`.

The end-to-end flow has five steps:

1. **Make a request to the model with tools it could call.**
   You send the conversation along with your tool definitions.
2. **Receive a tool call from the model.**
   The model responds with one or more tool calls instead of, or alongside, a text answer.
   IORails runs **tool call validation** here, before the tool calls reach your application.
3. **Execute code on the application side with input from the tool call.**
   Your application runs the function and produces a result.
4. **Make a second request to the model with the tool output.**
   You resend the conversation, including the assistant turn that made the calls and one tool result for each `tool_call_id`.
   IORails runs **tool result validation** here, before the model sees the results.
5. **Receive a final response from the model (or more tool calls).**
   The model returns a final text answer, or more tool calls that repeat the loop from step 2.

```mermaid
%%{init: {'theme': 'neutral', 'themeVariables': { 'background': 'transparent' }}}%%

sequenceDiagram
    participant App as Application
    participant GR as IORails
    participant LLM as Model

    App->>GR: 1. Request with tool definitions
    GR->>LLM: Forward request
    LLM-->>GR: 2. Tool calls
    Note over GR: tool call validation
    GR-->>App: Validated tool calls
    App->>App: 3. Execute the function
    App->>GR: 4. Request with tool results
    Note over GR: tool result validation
    GR->>LLM: Forward request
    LLM-->>GR: 5. Final response or more tool calls
    GR-->>App: Final response
```

## How Tool-Calling Rails Fit the Pipeline

Tool-calling rails mirror the input and output rails, with one rail per direction:

| Rail                   | Config section      | Flow name                | Direction     | What it checks                                                                                       |
| ---------------------- | ------------------- | ------------------------ | ------------- | ---------------------------------------------------------------------------------------------------- |
| Tool call validation   | `rails.tool_call`   | `tool call validation`   | Model output  | Tool calls the model emits: allowed name plus schema-valid arguments.                                |
| Tool result validation | `rails.tool_result` | `tool result validation` | Request input | Tool results sent back to the model: linkage to a prior call, name consistency, well-formed content. |

For each request, IORails runs the rails in the following order:

1. **Tool result validation** on the incoming messages (alongside input rails).
2. **Input rails**, then the **main LLM** call.
3. **Tool call validation** on the tool calls the model returned.
4. **Output rails** on the text response.
   A response that contains only tool calls and no text skips the text output rails.

When a rail blocks, IORails returns the refusal message `I'm sorry, I can't respond to that.` for a non-streaming request, and a `guardrails_violation` error payload for a streaming request.
When a rail cannot reach a verdict at all, because its provider rejected the credentials or was unreachable, the turn still stops, but the non-streaming message is `I'm sorry, an internal error has occurred.` instead.
That distinction lets a caller tell a provider outage, which is worth retrying, from a policy decision, which is not.

## Enable the IORails Engine

The built-in validation flows described on this page require the IORails engine.
Enable it in one of two ways.

Set the `NEMO_GUARDRAILS_IORAILS_ENGINE` environment variable so the top-level `LLMRails` import resolves to the IORails-backed engine.
This is the least invasive option for an existing application or the server:

```console
$ export NEMO_GUARDRAILS_IORAILS_ENGINE=1
```

```python
from nemoguardrails import LLMRails, RailsConfig

config = RailsConfig.from_path("./config")
rails = LLMRails(config)
```

Alternatively, construct the `Guardrails` facade directly, which uses the IORails engine by default:

```python
from nemoguardrails import Guardrails, RailsConfig

config = RailsConfig.from_path("./config")
rails = Guardrails(config)
```

If the configuration is not IORails-compatible, the `Guardrails` facade falls back to `LLMRails` and logs a warning.
Pass `require_iorails=True` to raise instead of falling back.
A configuration is IORails-compatible only when it uses Colang version `1.0`, declares no custom `llm` argument, and uses only IORails-supported rails and flows.

## Configure the Rails

Add the tool-calling flows to your `config.yml`.
When you use IORails, each `tool_*` section accepts only its own built-in flow name:

```yaml
models:
    - type: main
      engine: nim
      model: meta/llama-3.3-70b-instruct

rails:
    tool_call:
        flows:
            - tool call validation
    tool_result:
        flows:
            - tool result validation
```

You can enable either rail independently.
To validate only the model's tool calls, configure `rails.tool_call` and leave `rails.tool_result` empty.

> **Note**
>
> `rails.tool_output` and `rails.tool_input` are deprecated aliases for
> `rails.tool_call` and `rails.tool_result`, respectively.

> **Warning**
>
> The IORails flow names are fixed and direction-specific. `rails.tool_call`
> accepts only `tool call validation`, and `rails.tool_result` accepts only
> `tool result validation`. A misdirected, unknown, or duplicated tool flow name
> makes the configuration ineligible for IORails: the `Guardrails` facade
> silently falls back to `LLMRails`, and none of the IORails tool rails run. A
> typo in a flow name disables the tool rails you configured, with only a log
> warning to signal it. Pass `require_iorails=True` to the `Guardrails`
> constructor to raise instead of falling back silently. A configuration
> with `per_tool` rails always raises instead of falling back, and `LLMRails`
> issues a `UserWarning` when it loads one, because it does not run them.

> **Note**
>
> IORails honors the `parallel` field on the `tool_call` and `tool_result`
> sections, but `tool call validation` and `tool result validation` each
> name at most one flow, so `parallel` has no observable effect on them:
> they always run one at a time. `parallel` matters for per-tool checks
> configured under `tool_call.per_tool` / `tool_result.per_tool`, where a
> request can run several checks at once; refer to
> [Rail Engine Support](/nemo/guardrails/v0.25.0/reference/rail-engine-support) for per-tool
> configuration.

## Declare Tools

The rails validate against the tools you declare on the request.
Tool definitions use the provider-native OpenAI Chat Completions shape, and IORails forwards them to the model unchanged.

Declare tools per request in `options.llm_params`:

```python
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the weather for a city.",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        },
    },
]

response = await rails.generate_async(
    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
    options={"llm_params": {"tools": tools, "tool_choice": "auto"}},
)
```

When the model requests a tool, `generate_async` returns an assistant message that carries the tool calls in OpenAI shape:

```python
{
    "role": "assistant",
    "content": None,
    "tool_calls": [
        {
            "id": "call_1",
            "type": "function",
            "function": {"name": "get_weather", "arguments": "{\"city\": \"Paris\"}"},
        }
    ],
}
```

After your application executes the tool, send the result back to the model.
Resend the conversation with the original user message, the assistant turn that carried the tool call, and a `role: "tool"` message for each `tool_call_id`, then call `generate_async` again:

```python
messages = [
    {"role": "user", "content": "What's the weather in Paris?"},
    {
        "role": "assistant",
        "content": None,
        "tool_calls": [
            {
                "id": "call_1",
                "type": "function",
                "function": {"name": "get_weather", "arguments": "{\"city\": \"Paris\"}"},
            }
        ],
    },
    {
        "role": "tool",
        "tool_call_id": "call_1",
        "name": "get_weather",
        "content": "{\"temperature_c\": 18, \"condition\": \"cloudy\"}",
    },
]

response = await rails.generate_async(
    messages=messages,
    options={"llm_params": {"tools": tools}},
)
```

The `tool result validation` rail runs on this request before the model sees the result, and the model then returns a final text answer or more tool calls.

You can also declare tools statically on the model in `config.yml`.
IORails merges these into the set of tools the rails validate against, so the rails honor a config-declared tool even when a request carries no `llm_params`:

```yaml
models:
    - type: main
      engine: nim
      model: meta/llama-3.3-70b-instruct
      parameters:
          tools:
              - type: function
                function:
                    name: get_weather
                    description: Get the weather for a city.
                    parameters:
                        type: object
                        properties:
                            city:
                                type: string
                        required:
                            - city
```

## Tool Call Validation

The `tool call validation` rail inspects every tool call in the model's response and blocks the request when any call breaks a rule.

* **Allowlist.** The call must name a tool you declared.
  IORails blocks a call to a tool that is not in the declared set, for example `tool call 'delete_database' is not an allowed tool`.
* **Argument schema.** The call's arguments must validate against the tool's declared JSON Schema (the `function.parameters` block).
  IORails blocks arguments that violate the schema, for example `arguments for tool 'get_weather' do not match its schema: 'required' failed at '/city'`.
  The reason names the failing argument as the schema declares it, and the JSON Schema keyword.
  It never quotes an argument's value or a key the model chose, which appears as `*`.
* **No-argument tools.** A function tool that declares no parameters must receive no arguments.
  IORails blocks any supplied argument.
* **Hosted tools.** IORails allowlists a hosted or server-side tool that only `type` identifies (for example a built-in web search) by type, and does not schema-validate its arguments because the provider owns the call shape.

If the declared schema itself is not valid JSON Schema, the rail blocks the request rather than letting an unvalidated call through.

## Tool Result Validation

The `tool result validation` rail checks the tool results your application sends back to the model.
Because the OpenAI Chat Completions API is stateless, your application resends the conversation history, including the assistant turn that made the tool calls and the `role: "tool"` messages that answer them.
The rail validates each turn's results against that same turn's calls.

The rail blocks the request when a tool result:

* Is missing a `tool_call_id`, or that id does not correspond to a tool call earlier in the conversation.
* Reuses a `tool_call_id` that another result in the same turn already used.
  Each call must have exactly one result.
* Names a different tool than the call it links to.
* Carries content that is not a string or a list of content-block objects.

A tool result doesn't need a `name`.
The OpenAI tool message has none, so the rail identifies the tool from the result's `tool_call_id`.

> **Note**
>
> This rail validates structural well-formedness only. It confirms that the
> results are internally consistent with the calls in the request. It does not
> yet enforce a declared response schema, and it does not run a content-safety
> check on the tool result. Because the client resends the conversation, the
> linkage check verifies intra-request consistency, not server-verified
> provenance.

> **Warning**
>
> Every tool result must carry a `tool_call_id` that matches a tool call still
> present earlier in the same request's conversation history. If your
> application truncates or prunes older turns (for example a sliding context
> window), and a truncated turn held the call a later result answers, that
> result becomes unresolvable. With a per-tool policy enabled
> (`rails.tool_result.per_tool`), an unresolvable result fails closed and blocks
> the request, since which policy applies can no longer be verified.

## Control Rails per Request

Use `options.rails` to enable or disable each tool rail for a single request.
Each toggle accepts `true`, `false`, or a list of flow names, and defaults to `true`:

```python
response = await rails.generate_async(
    messages=messages,
    options={
        "llm_params": {"tools": tools},
        "rails": {"tool_call": True, "tool_result": False},
    },
)
```

## Check Tool Calls and Results Without Generation

An agent harness that calls its own model can use the same rails to validate tool traffic without IORails generating a response.
Run a check with `RailType.TOOL_CALL` on the conversation that ends in the model's tool calls, and with `RailType.TOOL_RESULT` after appending the tool results:

```python
from nemoguardrails import Guardrails, RailsConfig
from nemoguardrails.rails.llm.options import RailStatus, RailType

rails = Guardrails(RailsConfig.from_path("./config"), require_iorails=True)

call_check = await rails.check_async(messages, rail_types=[RailType.TOOL_CALL], tools=tools)
if call_check.status == RailStatus.BLOCKED:
    ...  # Drop or re-request the calls named in call_check.tool_violations.

messages.append({"role": "tool", "tool_call_id": "call_1", "content": "18C"})
result_check = await rails.check_async(messages, rail_types=[RailType.TOOL_RESULT])
```

A `tool_call` check validates the tool calls on the last assistant message against `tools`, which replace the tools declared on the main model.
Only a `tool_call` check reads `tools`, so passing them to any other check raises `InvalidCheckRequestError`, or returns HTTP 422 from the server.
Each `tools` entry must be a Chat Completions function tool, with a `function` object and a non-empty `name`; any other entry raises `InvalidCheckRequestError` rather than being dropped.
A tool check reads the messages and `tools` in the Chat Completions format whatever the `main` model's engine, and needs no `main` model.
The allowlist and argument schemas are enforced for every call by `tool call validation`, and for the calls a `per_tool` rail checks by that rail; `per_tool` rails check only the tools they are listed under.
A `tool_result` check validates every `tool` message against the calls it answers.
Both run the global validator and any `per_tool` rails, and neither calls the model.
A blocked check reports each failing call or result in `tool_violations`, with a `violation_type` to switch on.
For the full behavior, the violation types, and the engine requirements, refer to [Checking Messages Against Rails: Tool Rails](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-python-apis/check-messages#tool-rails-iorails-only).

> **Tip**
>
> To check the tool calls of a coding agent such as Claude Code or Codex from the agent's own hooks, use the [Guardrails hooks server](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-hooks-server/overview) (`nemoguardrails hooks-server`).
> It applies a tool policy of its own to each call, then runs the called tool's `per_tool` rails in this check.

The Guardrails API server (`nemoguardrails server`) exposes the same check through `/v1/checks`.
Run the server with `NEMO_GUARDRAILS_IORAILS_ENGINE=1` so the configuration is served by IORails; on `LLMRails`, the tool rail types return HTTP 422.
Pass the tools in the request's top-level `tools` field:

```json
{
  "model": "meta/llama-3.3-70b-instruct",
  "messages": [
    {"role": "user", "content": "What's the weather in Paris?"},
    {"role": "assistant", "content": null, "tool_calls": [
      {"id": "call_1", "type": "function", "function": {"name": "delete_files", "arguments": "{}"}}
    ]}
  ],
  "tools": [{"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}],
  "guardrails": {"config_id": "tools", "rail_types": ["tool_call"]}
}
```

The call names a tool that is not declared, so the response is:

```json
{
  "status": "blocked",
  "content": "I'm sorry, I can't respond to that.",
  "rail": "tool call validation",
  "reason": "tool call 'delete_files' is not an allowed tool",
  "tool_violations": [
    {"kind": "tool_call", "violation_type": "tool_not_allowed", "reason": "tool call 'delete_files' is not an allowed tool", "tool_call_id": "call_1", "tool_name": "delete_files", "index": 0}
  ]
}
```

## Streaming

Tool-calling rails work with `stream_async`.
IORails streams the text response as it arrives, accumulates the tool-call fragments, and runs `tool call validation` once the stream completes:

* On a clean stream, IORails emits the assembled tool calls as a final chunk after the text and output rails finish.
* When the rail blocks, IORails emits a `guardrails_violation` error payload and suppresses the tool-call chunk, so a consumer never receives a tool call after a block.
* A stream that contains only tool calls and no text skips the output rails.

## Limitations

* **Wire format.** In generation, the tool-calling rails support only the OpenAI Chat Completions shape, through the `openai` and `nim` engines.
  They do not yet support the OpenAI Responses API, Anthropic, Gemini, or Bedrock.
  A check takes messages and tools in the Chat Completions shape, whatever the `main` model's engine.
* **Structural result validation.** Tool result validation checks structure and linkage only.
  It does not enforce response schemas or apply content-safety checks to tool results.
* **Intra-request consistency.** The linkage check verifies consistency within a single request, not cross-turn provenance.
* **Schema dialect.** Argument validation uses JSON Schema.
  Tool schemas written in a different dialect are not validated against that dialect.
* **Engine requirement.** The built-in structural validation flows described on this page require the IORails engine and a Colang `1.0` configuration.

## Related Tool Features

The NVIDIA NeMo Guardrails library includes other tool-related capabilities that are distinct from the IORails tool-calling rails described here:

* [Tools Integration](/nemo/guardrails/v0.25.0/integration-with-third-party-libraries/tools-integration) covers LangChain tool calling and output-rail validation on the `LLMRails` engine.
* [Rail types](/nemo/guardrails/v0.25.0/about-nemo-guardrails-library/rail-types) describes Colang execution rails, which run actions before and after execution within the `LLMRails` event-driven pipeline.

The rails on this page operate at the request and response boundary of the IORails engine and do not execute tools.