Guardrails Configuration#

This section describes how to configure guardrails in the config.yml file to control LLM behavior.

The `rails` Key#

The rails key defines which guardrails are active and their configuration options. Rails are organized into five categories based on when they trigger during the guardrails process.

Rail Categories#

The following table summarizes the different rail categories and their trigger points.

Category	Trigger Point	Purpose
Input rails	When user input is received	Validate, filter, or modify user input
Retrieval rails	After RAG retrieval completes	Process retrieved chunks
Dialog rails	After canonical form is computed	Control conversation flow
Execution rails	Before/after action execution	Control custom action calls
Output rails	When LLM generates output	Validate, filter, or modify bot responses

The following diagram shows the guardrails process described in the table above in detail.

Diagram showing the programmable guardrails flow

Basic Configuration#

rails:
  input:
    flows:
      - self check input
      - jailbreak detection heuristics
      - mask sensitive data on input

  output:
    flows:
      - self check output
      - self check facts
      - check output sensitive data

  retrieval:
    flows:
      - check retrieval sensitive data

Input Rails#

Input rails process user messages before they reach the LLM:

rails:
  input:
    flows:
      - self check input                # LLM-based input validation
      - jailbreak detection heuristics  # Jailbreak detection
      - mask sensitive data on input    # PII masking

For a complete list of available input flows, refer to the Input Rails.

Output Rails#

Output rails process LLM responses before returning to users:

rails:
  output:
    flows:
      - self check output          # LLM-based output validation
      - self check facts           # Fact verification
      - self check hallucination   # Hallucination detection
      - mask sensitive data on output  # PII masking

For a complete list of available output flows, refer to the Output Rails.

Retrieval Rails#

Retrieval rails process chunks retrieved from the knowledge base:

rails:
  retrieval:
    flows:
      - check retrieval sensitive data

For a complete list of available retrieval flows, refer to the Retrieval Rails.

Dialog Rails#

Dialog rails control conversation flow after user intent is determined:

rails:
  dialog:
    single_call:
      enabled: false
      fallback_to_multiple_calls: true

    user_messages:
      embeddings_only: false

For a complete list of available dialog flows, refer to the Dialog Rails.

Execution Rails#

Execution rails control custom action and tool invocations:

rails:
  execution:
    flows:
      - check tool input
      - check tool output

Rail-Specific Configuration#

Configure options for specific rails using the config key:

rails:
  config:
    # Sensitive data detection settings
    sensitive_data_detection:
      input:
        entities:
          - PERSON
          - EMAIL_ADDRESS
          - PHONE_NUMBER
      output:
        entities:
          - PERSON
          - EMAIL_ADDRESS

    # Jailbreak detection settings
    jailbreak_detection:
      length_per_perplexity_threshold: 89.79
      prefix_suffix_perplexity_threshold: 1845.65

    # Fact-checking settings
    fact_checking:
      parameters:
        endpoint: "http://localhost:5000"

YAML Schema#

Complete guardrails configuration example:

rails:
  # Input validation
  input:
    flows:
      - self check input
      - jailbreak detection heuristics
      - mask sensitive data on input

  # Output validation
  output:
    flows:
      - self check output
      - self check facts

  # Retrieval processing
  retrieval:
    flows:
      - check retrieval sensitive data

  # Dialog behavior
  dialog:
    single_call:
      enabled: false

  # Rail-specific settings
  config:
    sensitive_data_detection:
      input:
        entities:
          - PERSON
          - EMAIL_ADDRESS
          - CREDIT_CARD
      output:
        entities:
          - PERSON
          - EMAIL_ADDRESS

Parallel Execution of Input and Output Rails#

You can configure input and output rails to run in parallel. This can improve latency and throughput.

IORails Engine#

The IORails engine is an optimized execution engine that runs NemoGuard input and output rails in parallel with dedicated model management. The IORails engine is an opt-in feature. By default, the NeMo Guardrails library uses the LLMRails engine.

Note

IORails is an early-release feature and currently does not support streaming, reasoning models, and telemetry as in LLMRails.

Supported Flows#

The IORails engine supports the following flows:

content safety check input / content safety check output
topic safety check input
jailbreak detection model

When IORails is enabled and the configuration uses only these flows, the engine runs them in parallel. Configurations that include custom flows, dialog rails, or other unsupported flows raise an error at initialization.

Enabling IORails#

To enable the IORails engine, set the NEMO_GUARDRAILS_IORAILS_ENGINE environment variable to 1:

NEMO_GUARDRAILS_IORAILS_ENGINE=1 nemoguardrails chat --config examples/configs/content_safety

When using the Python API, import the Guardrails class directly and pass use_iorails=True:

from nemoguardrails import RailsConfig
from nemoguardrails.guardrails.guardrails import Guardrails

config = RailsConfig.from_path("./config")
guardrails = Guardrails(config, use_iorails=True)

YAML-Based Parallel Execution#

You can also configure existing LLMRails flows to run in parallel using the parallel: True option in the config.yml file. This approach works with any flow type and does not require the IORails engine.

When to Use#

Use YAML-based parallel execution:

For I/O-bound rails such as external API calls to LLMs or third-party integrations.
If you have two or more independent input or output rails without shared state dependencies.
In production environments where response latency affects user experience and business metrics.

When Not to Use#

Avoid parallel execution:

For CPU-bound rails; it might not improve performance and can introduce overhead.
During development and testing for debugging and simpler workflows.

Configuration Example#

To enable parallel execution, set parallel: True in the rails.input and rails.output sections in the config.yml file.