Tools Integration with the NeMo Guardrails Library
This guide provides comprehensive instructions for integrating and using tools within the NeMo Guardrails library via the LLMRails interface. It covers supported tools, configuration settings, practical examples, and important security considerations for safe and effective implementation.
Overview
The NeMo Guardrails library supports the integration of tools to enhance the capabilities of language models while maintaining safety controls. Tools can be used to extend the functionality of your AI applications by enabling interaction with external services, APIs, databases, and custom functions.
For a diagram of the request and tool-call loop and the points where tool rails validate it, see How Function Calling Works.
Supported Version
Tool calling is available starting from the NeMo Guardrails library version 0.17.0.
Supported Tools
The NeMo Guardrails library supports LangChain tools, which provide a standardized interface for integrating external functionality into language model applications.
LangChain Tools
The NeMo Guardrails library is fully compatible with LangChain tools, including:
- Built-in LangChain Tools: Weather services, calculators, web search, database connections, and more
- Community Tools: Third-party tools available in the LangChain ecosystem
- Custom Tools: User-defined tools created using the LangChain tool interface
Creating Custom Tools
You can create custom tools by following the LangChain documentation patterns. Here’s an example:
For detailed information on creating custom tools, refer to the LangChain Tools Documentation.
Tool Call Schema
The NVIDIA NeMo Guardrails library does not define its own tool-call schema, but the shape you get back depends on how you call it.
RunnableRails converts tool calls into LangChain’s ToolCall TypedDict shape:
Calling rails.generate() directly does not go through this conversion. response.tool_calls retains the model-native, OpenAI-style function-call structure instead:
If your code reads tool_call["name"] or tool_call["args"] against a direct rails.generate() response, it is reading the wrong keys; use tool_call["function"]["name"] and tool_call["function"]["arguments"] instead, or wrap the model with RunnableRails to get the normalized LangChain shape.
When the model response contains tool calls, the library runs any tool call rails configured under rails.tool_call, which can block the calls, instead of the output rails.
Only approved calls are returned for execution. If a tool call rail blocks the request, the response contains the configured refusal and omits executable tool calls. If a rail changes the calls, the returned calls and cached conversation history use the approved values. Without such changes, the library preserves tool-call names and arguments, apart from parsing a JSON arguments string into a dictionary.
Configuration Settings
Passthrough Mode and Dialog Flows
With Colang 1.0, tool calls are supported with or without passthrough when the guardrails configuration has no user-intent definitions (user_messages). Configurations with user-intent definitions drop tool calls and return only the generated text. Enabling rails.dialog.single_call.enabled alone does not block tool calls.
The /v1/chat/completions endpoint accepts tools, tool_choice, and parallel_tool_calls for Colang 1.0 configurations without user-intent definitions. Unsupported combinations return HTTP 422 with an OpenAI-compatible error object whose type is invalid_request_error. Colang 2.x configurations do not propagate provider tool calls through this endpoint. With stream: true, the server completes generation and enabled rail checks before delivering the approved answer as successive SSE deltas. Tool-argument fragments are emitted only after the complete call passes validation. See Server Tool Calling.
For a tool round trip, append the returned assistant tool calls and a role: tool result for each call, preserving its tool_call_id. Results can arrive in a different order from the calls. The next generation can return another set of tool calls or a final answer. Your model and provider must support the requested tool options.
When you pass generation options to LLMRails.generate_async, the returned GenerationResponse stores tool calls in its tool_calls field. Include that field when building the assistant history message:
For a custom chat general prompt, include {{ history | to_chat_messages }} to retain native assistant and tool roles. The default text prompt also preserves tool calls and results as native messages after its initial rendered instructions and conversation prefix.
If your configuration does not use dialog flows, using passthrough mode is still recommended because:
- Internal NeMo Guardrails library tasks do not require tool use and might provide erroneous results if tools are enabled
- It ensures that the LLM can properly handle tool calls and responses
- It maintains the natural flow of tool-based conversations
Configure passthrough mode in your configuration:
The key differences between configurations:
- bare_config: No rails at all, pure LLM with passthrough
- unsafe_config: Only has input rails, tool results bypass validation
- safe_config: Has both input and output rails for complete protection
We will use these configurations in the examples below.
Tool Result Rails
Configure rails.tool_result.flows to validate tool results before the next model call. Each result in the latest batch passes through the configured flows. A flow can inspect $tool_message, $tool_name, and $tool_call_id, and replace $tool_message with sanitized content. When a tool result omits name, as server requests do, the library resolves it from the matching assistant call.
After all results pass, generation resumes using the approved content. If a flow blocks a result with abort, generation stops and returns its refusal without calling the main model. In subsequent turns using the cached event history, unapproved results are replaced by [Tool result blocked by guardrails.]. Tool-result rails apply in both passthrough modes; input rails process user messages, and output rails process the generated textual answer. Setting options.rails.tool_result to false explicitly bypasses tool-result validation for that request.
The library leaves caller-supplied messages unchanged. Sanitized values are retained in the LLMRails event history, so the raw input list is not a sanitized transcript. Passthrough prompts also retain earlier input-rail rewrites from that history; blocked user turns become [User message blocked by guardrails.]. If you reconstruct history independently of that event history, preserve approved content yourself rather than replaying raw results as already validated history.
Prompt Length for Tool Conversations
For Colang 1.0 LLMRails configurations with passthrough: false, the selected general prompt’s max_length applies to the complete assembled tool conversation. Both text and chat prompt templates preserve tool calls and their results without automatically trimming history or truncating tool data.
The limit counts characters in the JSON-serialized prompt messages, including instructions, tool arguments, results, and message metadata. For this calculation, inline base64 image URLs in structured message content are replaced with [IMAGE_CONTENT]; the original image data is still sent to the model. Text content and tool arguments count in full, even if they contain image data URLs. The limit is separate from the model provider’s token context limit and does not measure the entire provider request, such as the tools definitions. Task prompts default to 16000 characters unless configured otherwise.
If the conversation exceeds the limit, generation raises ToolConversationTooLongError before calling the generation model. The /v1/chat/completions endpoint returns HTTP 422 before any SSE delivery with an OpenAI-compatible error object: type: invalid_request_error, code: tool_conversation_too_long, and param: messages. The error reports the measured size and configured limit without including conversation content.
Reduce the tool-result size or remove complete older conversation turns, keeping tool calls and their matching results together. Alternatively, increase max_length on your selected general prompt within your model’s context capacity. Setting max_length: null disables this local length check; the provider’s context limit still applies.
Text-only conversations retain the existing history-trimming behavior. Passthrough mode does not render the general prompt, so this prompt-length check does not apply to passthrough requests.
Implementation Examples
Example 1: Multi-Tool Implementation
This example demonstrates how to implement multiple tools with proper tool call handling:
Example 2: Single-Call Tool Processing
This example shows how to handle pre-processed tool results:
Security Considerations
Tool Message Risks
Important: Tool messages are not subject to input rails validation. This presents potential security risks:
- Tool responses may contain unsafe content that bypasses input guardrails
- Malicious or unexpected tool outputs could influence the model’s responses
- Tool execution results are trusted by default
Recommended Safety Measures
Use tool-result rails to validate or sanitize tool results before the model receives them, and output rails to validate the generated answer.
Tool Security: Unsafe Content in Tool Results
The Problem: Tool Results Bypass Input Rails
Tool messages are not subject to input rails validation, creating a security vulnerability where unsafe tool results can bypass guardrails and influence the LLM’s responses.
Demonstration: Bare LLM vs Rails Configuration
Expected Results
The results may vary slightly due to the non-deterministic nature of LLMs, but typically:
- Bare Config Output:
"Client ID BOMB ME — NVIDIA (NVDA) stock price: $180.00." - Unsafe Config Output:
"Client ID BOMB ME — NVIDIA (NVDA) stock price: $180.00." - Safe Config Output:
"I'm sorry, I can't respond to that."
LangChain and LangGraph Integration
For integration with LangChain and LangGraph workflows, refer to:
- LangChain Integration Guide - Using RunnableRails to add guardrails to LangChain chains
- LangGraph Integration Guide - Tool calling with LangGraph stateful multi-agent workflows