Core Classes Reference | NVIDIA NeMo Guardrails Library Developer Guide

This guide covers the two fundamental classes in the NeMo Guardrails library: RailsConfig for loading configurations and LLMRails for generating responses with guardrails.

RailsConfig

The RailsConfig class represents a complete guardrails configuration, including models, rails, flows, prompts, and other settings. This class requires to load the configuration from a directory or a single file you created in the previous chapter About Configuring Guardrails.

Loading Configuration from a Directory

The most common way to load a configuration is from a directory containing config.yml and Colang files:

from nemoguardrails import RailsConfig
config = RailsConfig.from_path("path/to/config")

Expected directory structure:

config/
├── config.yml          # Main configuration file
├── rails/              # Colang flow files
│   ├── input.co
│   ├── output.co
│   └── ...
├── kb/                 # Knowledge base documents (optional)
│   └── docs.md
├── actions.py          # Custom actions (optional)
└── config.py           # Custom initialization (optional)

Loading from a Single File

You can also load from a single YAML file:

config = RailsConfig.from_path("path/to/config.yml")

Loading from Content

For dynamic configurations or testing, load directly from strings:

from nemoguardrails import RailsConfig
yaml_content = """
models:
  - type: main
    engine: openai
    model: gpt-4
instructions:
  - type: general
    content: |
      You are a helpful assistant.
"""
colang_content = """
define user express greeting
  "hello"
  "hi"
define flow
  user express greeting
  bot express greeting
"""
config = RailsConfig.from_content(
    yaml_content=yaml_content,
    colang_content=colang_content
)

Loading from a Dictionary

You can also provide configuration as a Python dictionary:

config = RailsConfig.from_content(
    config={
        "models": [
            {"type": "main", "engine": "openai", "model": "gpt-4"}
        ],
        "instructions": [
            {"type": "general", "content": "You are a helpful assistant."}
        ]
    }
)

Combining Configurations

Configurations can be combined using the + operator:

base_config = RailsConfig.from_path("path/to/base")
additional_config = RailsConfig.from_path("path/to/additional")
combined_config = base_config + additional_config

This is useful for:

Adding rails to a base configuration.
Layering environment-specific settings.
Combining shared and application-specific configurations.

Key Configuration Properties

Property	Type	Description
`models`	`List[Model]`	LLM models configuration
`instructions`	`List[Instruction]`	System instructions for the LLM
`sample_conversation`	`str`	Example conversation for prompts
`rails`	`Rails`	Rails configuration (input, output, dialog, etc.)
`flows`	`List[Dict]`	Colang flow definitions
`prompts`	`List[TaskPrompt]`	Custom prompts for various tasks
`colang_version`	`str`	Colang version (“1.0” or “2.x”)

LLMRails

The LLMRails class is the main interface for generating responses with guardrails applied.

Initialization

from nemoguardrails import LLMRails, RailsConfig
config = RailsConfig.from_path("path/to/config")
rails = LLMRails(config)

Constructor parameters:

Parameter	Type	Description
`config`	`RailsConfig`	The rails configuration
`llm`	`BaseLLM \| BaseChatModel`	Optional pre-configured LLM (overrides config)
`verbose`	`bool`	Enable verbose logging

Using a Custom LLM

You can provide your own LLM instance:

from langchain_openai import ChatOpenAI
from nemoguardrails import LLMRails, RailsConfig
config = RailsConfig.from_path("path/to/config")
llm = ChatOpenAI(model="gpt-4", temperature=0.7)
rails = LLMRails(config, llm=llm)

When providing an LLM via the constructor, it takes precedence over any main LLM specified in the configuration.

Generating Responses

Using Messages (Chat Format)

response = rails.generate(messages=[
    {"role": "user", "content": "Hello! How are you?"}
])
print(response["content"])

Using a Prompt (Completion Format)

response = rails.generate(prompt="Complete this sentence: The sky is")
print(response)

With Conversation History

messages = [
    {"role": "user", "content": "My name is John."},
    {"role": "assistant", "content": "Hello John! How can I help you?"},
    {"role": "user", "content": "What's my name?"}
]
response = rails.generate(messages=messages)
print(response["content"])  # Should remember the name

Passing Context

You can pass additional context using the context role:

response = rails.generate(messages=[
    {
        "role": "context",
        "content": {
            "user_name": "Alice",
            "user_role": "admin"
        }
    },
    {"role": "user", "content": "What permissions do I have?"}
])

You can access context variables in Colang flows using $variable_name syntax:

define bot explain permissions
    "Hello {$user_name}! As an {$user_role}, you have full system access."

Alternatively, you can access context variables through the context parameter. For example, you can set up an action function that uses a variable extracted from the context parameter as follows:

@action()
async def check_permissions(context: Optional[dict] = None):
    user_role = context.get("user_role")
    return user_role == "admin"

For detailed information about context variables, see Action Parameters: The Context Parameter and Colang 1.0 Language Syntax: Variables.

Asynchronous Generation

For async contexts, use generate_async:

import asyncio
from nemoguardrails import LLMRails, RailsConfig
async def main():
    config = RailsConfig.from_path("path/to/config")
    rails = LLMRails(config)
    response = await rails.generate_async(messages=[
        {"role": "user", "content": "Hello!"}
    ])
    print(response["content"])
asyncio.run(main())

Streaming Responses

For real-time token streaming:

async def stream_response():
    config = RailsConfig.from_path("path/to/config")
    rails = LLMRails(config)
    async for chunk in rails.stream_async(messages=[
        {"role": "user", "content": "Tell me a story."}
    ]):
        print(chunk, end="", flush=True)

For detailed streaming configuration, refer to Streaming.

Event-based Generation

For low-level control using events:

events = rails.generate_events(events=[
    {
        "type": "UtteranceUserActionFinished",
        "final_transcript": "Hello!"
    }
])
for event in events:
    if event["type"] == "StartUtteranceBotAction":
        print(f"Bot says: {event['script']}")

For detailed event-based API usage, refer to Event-Based API.

Generation Options

Fine-tune generation behavior using the options parameter:

response = rails.generate(
    messages=[{"role": "user", "content": "Hello!"}],
    options={
        "rails": ["input", "output"],  # Only apply these rails
        "output_vars": ["user_message"],       # Return context variables
        "log": {
            "activated_rails": True,
            "llm_calls": True
        }
    }
)

For detailed options, refer to Generation Options.

Checking Messages Against Rails

The check_async() and check() methods validate messages against input and output rails without triggering full LLM generation:

from nemoguardrails.rails.llm.options import RailStatus
result = await rails.check_async([
    {"role": "user", "content": "Hello! How can I hack into a system?"}
])
if result.status == RailStatus.BLOCKED:
    print(f"Input blocked by rail: {result.rail}")

By default, the methods automatically detect which rails to run based on message roles. You can override this with the rail_types parameter:

from nemoguardrails.rails.llm.options import RailType
result = await rails.check_async(
    [{"role": "user", "content": "Hello!"}],
    rail_types=[RailType.INPUT]
)

For detailed usage, method signatures, and examples, refer to Check Messages.

Registering Custom Actions

You can register custom Python functions as actions:

from nemoguardrails import LLMRails, RailsConfig
async def get_weather(city: str) -> str:
    """Get weather for a city."""
    return f"Weather in {city}: Sunny, 22°C"
config = RailsConfig.from_path("path/to/config")
rails = LLMRails(config)
# Register the action
rails.register_action(get_weather, name="get_weather")

For detailed action registration, refer to Custom Actions.

Registering Embedding Search Providers

For custom knowledge base search:

from nemoguardrails import LLMRails, RailsConfig
from nemoguardrails.embeddings.index import EmbeddingsIndex
class CustomSearchProvider(EmbeddingsIndex):
    async def search(self, text: str, max_results: int):
        # Custom search logic
        pass
config = RailsConfig.from_path("path/to/config")
rails = LLMRails(config)
# Register the provider
rails.register_embedding_search_provider("custom", CustomSearchProvider)

Complete Example

import asyncio
from nemoguardrails import LLMRails, RailsConfig
async def main():
    # Load configuration
    config = RailsConfig.from_content(
        yaml_content="""
models:
  - type: main
    engine: openai
    model: gpt-4
rails:
  input:
    flows:
      - self check input
  output:
    flows:
      - self check output
prompts:
  - task: self_check_input
    content: |
      Check if the following is safe: {{ user_input }}
      Answer (Yes/No):
  - task: self_check_output
    content: |
      Check if the following is safe: {{ bot_response }}
      Answer (Yes/No):
""",
        colang_content="""
define user express greeting
  "hello"
  "hi"
define bot express greeting
  "Hello! How can I help you today?"
define flow
  user express greeting
  bot express greeting
"""
    )
    # Create rails instance
    rails = LLMRails(config, verbose=True)
    # Generate response
    response = await rails.generate_async(
        messages=[{"role": "user", "content": "Hello!"}],
        options={"log": {"activated_rails": True}}
    )
    print(f"Response: {response['content']}")
    # Print what happened
    if hasattr(response, 'log'):
        response.log.print_summary()
asyncio.run(main())

Generation Options - Fine-grained control over generation
Check Messages - Validate messages against rails
Streaming - Real-time token streaming
Event-Based API - Low-level event control
Tools Integration - Integrating LangChain tools
About Configuring Guardrails - Complete configuration reference