RunnableRails

This guide demonstrates how to integrate the NeMo Guardrails library into LangChain applications using the RunnableRails class. The class implements the full Runnable Protocol with comprehensive support for synchronous and asynchronous operations, streaming, and batch processing.

Overview

RunnableRails provides a complete LangChain-native interface that wraps guardrail configurations around LLMs or entire chains. It supports all Runnable methods including invoke(), ainvoke(), stream(), astream(), batch(), and abatch() with full metadata preservation.

Getting Started

To get started, load a guardrail configuration and create a RunnableRails instance.

1
from nemoguardrails import RailsConfig
2
from nemoguardrails.integrations.langchain.runnable_rails import RunnableRails
3
4
config = RailsConfig.from_path("path/to/config")
5
guardrails = RunnableRails(config)

To add guardrails around an LLM model inside a chain, wrap the LLM model with a RunnableRails instance. For example, (guardrails | ...).

The following is an example of using a prompt, model, and output parser:

1
from langchain_openai import ChatOpenAI
2
from langchain_core.prompts import ChatPromptTemplate
3
from langchain_core.output_parsers import StrOutputParser
4
5
prompt = ChatPromptTemplate.from_template("tell me a short joke about {topic}")
6
model = ChatOpenAI()
7
output_parser = StrOutputParser()
8
9
chain = prompt | model | output_parser

Add guardrails around the LLM model in the example above with the following code:

1
chain_with_guardrails = prompt | (guardrails | model) | output_parser

To add guardrails to an existing chain or any Runnable, wrap it similarly.

1
rag_chain = (
2
    {"context": retriever | format_docs, "question": RunnablePassthrough()}
3
    | prompt
4
    | llm
5
    | StrOutputParser()
6
)
7
8
rag_chain_with_guardrails = guardrails | rag_chain

You can also use the same approach to add guardrails only around certain parts of your chain. The following example from the RunnableBranch Documentation adds guardrails around the "anthropic" and "general" branches inside a RunnableBranch.

1
from langchain_core.runnables import RunnableBranch
2
3
branch = RunnableBranch(
4
    (lambda x: "anthropic" in x["topic"].lower(), guardrails | anthropic_chain),
5
    (lambda x: "langchain" in x["topic"].lower(), langchain_chain),
6
    guardrails | general_chain,
7
)

In general, you can wrap any part of a runnable chain with guardrails.

1
chain = runnable_1 | runnable_2 | runnable_3 | runnable_4 | ...
2
chain_with_guardrails = runnable_1 | (guardrails | (runnable_2 | runnable_3)) | runnable_4 | ...

Streaming Support

RunnableRails provides full streaming support with both synchronous and asynchronous methods. This enables responsive applications that stream LLM outputs as they are generated.

1
# Synchronous streaming
2
for chunk in guardrails.stream("What is machine learning?"):
3
    print(chunk, end="", flush=True)
4
5
# Asynchronous streaming
6
async def stream_example():
7
    async for chunk in guardrails.astream("What is machine learning?"):
8
        print(chunk, end="", flush=True)

Metadata in Streaming: RunnableRails preserves all metadata during streaming, including response_metadata, usage_metadata, and additional_kwargs in AIMessageChunk objects.

Batch Processing

RunnableRails supports efficient batch processing for multiple inputs. The following example shows how to use the batch and abatch methods.

1
inputs = [
2
    "What is Python?",
3
    "Explain machine learning",
4
    "How does AI work?"
5
]
6
7
# Synchronous batch processing
8
results = guardrails.batch(inputs)
9
10
# Asynchronous batch processing
11
results = await guardrails.abatch(inputs)
12
13
# Control concurrency
14
from langchain_core.runnables import RunnableConfig
15
config = RunnableConfig(max_concurrency=3)
16
results = await guardrails.abatch(inputs, config=config)

Input/Output Formats

RunnableRails intelligently handles various input and output formats with automatic transformation.

LLM Wrapping Formats

Chain Wrapping Formats

Metadata Preservation

RunnableRails maintains complete metadata compatibility with LangChain components. All AIMessage responses include the following:

• response_metadata: Token usage, model info, finish reasons.
• usage_metadata: Input/output token counts, total tokens.
• additional_kwargs: Custom fields from the LLM provider.
• id: Unique message identifiers.
• tool_calls: Tool call information when applicable.

1
result = guardrails.invoke("Hello world")
2
print(result.response_metadata)  # {'token_usage': {...}, 'model_name': '...', ...}
3
print(result.usage_metadata)     # {'input_tokens': 10, 'output_tokens': 5, ...}
4
print(result.additional_kwargs)  # Provider-specific fields
5
print(result.id)                 # 'msg_abc123...'

This ensures seamless integration with LangChain components that depend on message metadata.

Configuration Options

Passthrough Mode

The role of a guardrail configuration is to validate user input, check LLM output, and guide the LLM model on how to respond. See the Configuration Guide for more details on the different types of rails.

To achieve this, the guardrail configuration might make additional calls to the LLM or other models/APIs (for example, for fact-checking and content moderation).

By default, when the guardrail configuration decides that it is safe to prompt the LLM, it uses the exact prompt that was provided as the input, such as a string, StringPromptValue or ChatPromptValue. However, to enforce specific rails, for example, dialog rails, general instructions, the guardrails configuration needs to alter the prompt used to generate the response.

The passthrough parameter controls this behavior.

• passthrough=True (default): Uses the exact input prompt with minimal guardrail intervention.
• passthrough=False: Allows guardrails to modify prompts for enhanced protection.

1
# Minimal intervention (required for tool calling)
2
guardrails = RunnableRails(config, passthrough=True)
3
4
# Enhanced guardrails (modifies prompts as needed)
5
guardrails = RunnableRails(config, passthrough=False)

Tool Calling Requirement: Set passthrough=True for proper tool call handling.

Custom Input/Output Keys

When you use a guardrail configuration to wrap a chain or a Runnable, the input and output are either dictionaries or strings. However, a guardrail configuration always operates on a text input from the user and a text output from the LLM. To achieve this, when dictionaries are used, one of the keys from the input dictionary must be designated as the "input text" and one of the keys from the output as the "output text".

By default, these keys are input and output. To customize these keys, provide the input_key and output_key parameters when creating the RunnableRails instance.

The following examples show how to customize the input and output keys with “question” and “answer” keys.

1
# Custom keys for specialized chains
2
guardrails = RunnableRails(
3
    config,
4
    input_key="question",    # Default: "input"
5
    output_key="answer"      # Default: "output"
6
)
7
8
# Usage with RAG chain
9
rag_chain_with_guardrails = guardrails | rag_chain

When a guardrail is triggered and predefined messages must be returned instead of the output from the LLM, only a dictionary with the output key is returned.

1
{"answer": "I can't assist with that request."}

Tool Calling

RunnableRails supports LangChain tool calling with full metadata preservation and streaming. Tool calling requires passthrough=True to work properly.

The following steps are required to use tool calling with RunnableRails:

• Set passthrough=True when creating RunnableRails instance.
• Use bind_tools() to attach tools to your model.
• Handle tool execution in your application logic.

Basic Tool Setup

1
from langchain_core.tools import tool
2
from langchain_openai import ChatOpenAI
3
from nemoguardrails import RailsConfig
4
from nemoguardrails.integrations.langchain.runnable_rails import RunnableRails
5
6
@tool
7
def calculator(expression: str) -> str:
8
    """Evaluates mathematical expressions like '2 + 2' or 'sqrt(16)'."""
9
    try:
10
        safe_dict = {'sqrt': __import__('math').sqrt, 'pow': pow, '__builtins__': {}}
11
        return str(eval(expression, safe_dict))
12
    except Exception as e:
13
        return f"Error: {e}"
14
15
tools = [calculator]
16
model = ChatOpenAI(model="gpt-5").bind_tools(tools)
17
config = RailsConfig.from_path("path/to/config")
18
guardrails = RunnableRails(config=config, passthrough=True)
19
guarded_model = guardrails | model

Two-Call Tool Pattern

The standard flow for two-call tool calling is to get tool calls, execute them, and synthesize results.

1
from langchain_core.messages import AIMessage, HumanMessage, ToolMessage
2
3
# First call: Get tool calls
4
messages = [HumanMessage(content="What is 2 + 2?")]
5
result = guarded_model.invoke(messages)
6
7
# Execute tools
8
tools_by_name = {tool.name: tool for tool in tools}
9
messages_with_tools = [
10
    messages[0],
11
    AIMessage(content=result.content or "", tool_calls=result.tool_calls),
12
]
13
14
for tool_call in result.tool_calls:
15
    tool_result = tools_by_name[tool_call["name"]].invoke(tool_call["args"])
16
    messages_with_tools.append(
17
        ToolMessage(
18
            content=str(tool_result),
19
            name=tool_call["name"],
20
            tool_call_id=tool_call["id"],
21
        )
22
    )
23
24
# Second call: Synthesize results
25
final_result = guarded_model.invoke(messages_with_tools)
26
print(final_result.content)

Single-Call with Pre-processed Messages

Use single-call tool calling when you already have a complete message history with tool results.

1
messages = [
2
    HumanMessage(content="What is 2 + 2?"),
3
    AIMessage(
4
        content="",
5
        tool_calls=[
6
            {
7
                "name": "calculator",
8
                "args": {"expression": "2 + 2"},
9
                "id": "call_001",
10
                "type": "tool_call",
11
            }
12
        ],
13
    ),
14
    ToolMessage(
15
        content="4",
16
        name="calculator",
17
        tool_call_id="call_001",
18
    ),
19
]
20
21
result = guarded_model.invoke(messages)
22
print(result.content)  # "2 + 2 equals 4."

Composition and Chaining

RunnableRails integrates with complex LangChain compositions. The following example shows how to use RunnableRails with a conditional branching chain.

1
from langchain_core.runnables import RunnablePassthrough, RunnableBranch
2
from langchain_core.output_parsers import StrOutputParser
3
4
# Complex chain with guardrails
5
chain = (
6
    {"context": retriever | format_docs, "question": RunnablePassthrough()}
7
    | prompt
8
    | (guardrails | llm)
9
    | StrOutputParser()
10
)
11
12
# Conditional branching with guardrails
13
branch = RunnableBranch(
14
    (lambda x: "technical" in x["topic"], guardrails | technical_chain),
15
    (lambda x: "creative" in x["topic"], creative_chain),
16
    guardrails | general_chain,
17
)

Key Benefits of RunnableRails:

• Maintains full Runnable protocol compatibility.
• Preserves metadata throughout the chain.
• Supports all async/sync operations.
• Works with streaming and batch processing.