Skip to navigation

Overview of the NeMo Guardrails Python APIs

View as MarkdownOpen in Claude

The NeMo Guardrails library Python API provides two core classes for running guardrails:

  • RailsConfig: Loads and manages guardrails configuration from files or content.
  • Guardrails: The main interface for generating responses with guardrails applied. It runs the IORails engine when that engine supports the configuration; otherwise it falls back to the deprecated LLMRails, unless you pass require_iorails=True or the configuration has per-tool rails, in which case it raises ValueError.

Upon initializing the core classes (RailsConfig and Guardrails), the library loads the configuration files you created in the previous chapter About Configuring Guardrails.

LLMRails, the earlier main interface, is deprecated in 0.25.0; removal is planned no earlier than 0.27.0. Replace LLMRails(config) with Guardrails(config). For the steps, refer to Migrating to 0.25.

Quick Start

The following steps show how to run a sample guardrailed chat request using the NeMo Guardrails library Python API.

Prerequisites

Meet the following prerequisites to use the NeMo Guardrails library Python API.

  1. If you haven’t already, install the NeMo Guardrails library with the nvidia extra, following the instructions in Installation.

  2. Set up an environment variable for your NVIDIA API key.

    export NVIDIA_API_KEY="your-nvidia-api-key"

    This is required to access NVIDIA-hosted models on build.nvidia.com. The provided example configurations (examples/configs) and code examples throughout the documentation use NVIDIA-hosted models.

Run a Sample Guardrailed Chat Request

The following example shows the minimal code to load the content safety example configuration and generate a response using the Guardrails class.

from nemoguardrails import Guardrails, RailsConfig
# Load configuration from the config directory
config = RailsConfig.from_path("examples/configs/content_safety")
# Create the Guardrails instance
rails = Guardrails(config)
# Generate a response
response = rails.generate(messages=[
{
"role": "user",
"content": "What is the capital of France?"
}
])
print(response["content"])

Lifetime and Performance

Construct Guardrails once per process and reuse it across requests. Construction validates the configuration and compiles the configured rails.

The synchronous generate() and check() methods start a short-lived engine for every call, with tracing and metrics turned off. They suit scripts. In a service, start the engine once and call the asynchronous methods. Starting the engine opens an HTTP session for each configured model and starts the request queue, and later generate_async(), stream_async(), and check_async() calls reuse them.

from nemoguardrails import Guardrails, RailsConfig
async def main():
config = RailsConfig.from_path("./config")
async with Guardrails(config) as rails:
response = await rails.generate_async(messages=[{"role": "user", "content": "Hello!"}])
print(response["content"])

If your application has its own startup and shutdown hooks, call await rails.startup() and await rails.shutdown() from them instead of using async with.

For serverless or FaaS handlers, create the Guardrails instance in module scope so that warm invocations skip loading and validating the configuration:

from nemoguardrails import Guardrails, RailsConfig
_rails = Guardrails(RailsConfig.from_path("./config"))
def handler(event, context):
return _rails.generate(messages=event["messages"])

The synchronous generate() still starts a short-lived engine for each invocation. If your platform keeps one event loop running across invocations, make the handler asynchronous and call await _rails.generate_async(...) instead, so that warm invocations reuse the running engine.

When to Use Each API

APIUse Case
generate() / generate_async()Standard chat interactions with messages
stream_async()Real-time token streaming
check() / check_async()Validate messages against rails without generation
generate_events() / generate_events_async()Low-level event control for custom integrations (LLMRails only)

Synchronous vs Asynchronous

The NeMo Guardrails library provides both synchronous and asynchronous methods:

SynchronousAsynchronousDescription
generate()generate_async()Generate responses from messages
check()check_async()Run rails on messages, without generation
generate_events()generate_events_async()Generate events from event history
-stream_async()Stream tokens asynchronously

Use asynchronous methods (generate_async(), stream_async(), check_async()) in async contexts for better performance. The synchronous generate() and check() methods cannot be called from within an async context.