Regular Expression Detection Integration

The NVIDIA NeMo Guardrails library provides out-of-the-box support for content moderation based on regular expression (regex) pattern matching. This integration enables you to detect and block content that matches specific patterns in user inputs, bot outputs, and retrieved knowledge base chunks.

Overview

Regex detection is useful for scenarios such as:

• Blocking specific keywords or phrases
• Detecting patterns like Social Security Numbers, credit card numbers, or other sensitive data formats
• Filtering profanity or inappropriate content using basic regex pattern matching

Setup

No additional packages are required. The regex detection rails is built into the NVIDIA NeMo Guardrails library.

Usage

The regex detection rails uses a flexible configuration that allows you to define specific regex patterns for input, output, and retrieval checking.

Configuration Structure

Add the regex configuration to your config.yml file:

1
rails:
2
  config:
3
    regex_detection:
4
      input:
5
        patterns:
6
          - "\\b(password|secret|api[_-]?key)\\b"
7
          - "\\d{3}-\\d{2}-\\d{4}"  # SSN pattern
8
        case_insensitive: true
9
      output:
10
        patterns:
11
          - "\\bconfidential\\b"
12
          - "\\binternal[_-]?use[_-]?only\\b"
13
        case_insensitive: true
14
      retrieval:
15
        patterns:
16
          - "\\bclassified\\b"
17
        case_insensitive: false
18
  input:
19
    flows:
20
      - regex check input
21
  output:
22
    flows:
23
      - regex check output
24
  retrieval:
25
    flows:
26
      - regex check retrieval

Configuration Options

Input Rails

To detect regex patterns in user input:

1
rails:
2
  config:
3
    regex_detection:
4
      input:
5
        patterns:
6
          - "\\b(hack|exploit|bypass)\\b"
7
        case_insensitive: true
8
  input:
9
    flows:
10
      - regex check input

Output Rails

To detect regex patterns in bot output:

1
rails:
2
  config:
3
    regex_detection:
4
      output:
5
        patterns:
6
          - "\\bdo not share\\b"
7
        case_insensitive: true
8
  output:
9
    flows:
10
      - regex check output

Retrieval Rails

When a chunk matches any of the configured retrieval patterns, that chunk is removed from the retrieval results and is not passed to the model. Only chunks that do not match the patterns are kept and used as context.

To detect regex patterns in retrieved knowledge base chunks:

1
rails:
2
  config:
3
    regex_detection:
4
      retrieval:
5
        patterns:
6
          - "\\bproprietary\\b"
7
        case_insensitive: true
8
  retrieval:
9
    flows:
10
      - regex check retrieval

Complete Example

Here’s a comprehensive example configuration:

1
models:
2
  - type: main
3
    engine: openai
4
    model: gpt-4
5
6
rails:
7
  config:
8
    regex_detection:
9
      input:
10
        patterns:
11
          # Block sensitive data patterns
12
          - "\\d{3}-\\d{2}-\\d{4}"          # SSN
13
          - "\\d{4}[- ]?\\d{4}[- ]?\\d{4}[- ]?\\d{4}"  # Credit card
14
          - "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}"  # Email
15
          # Block specific keywords
16
          - "\\b(password|secret|api[_-]?key|token)\\b"
17
        case_insensitive: true
18
      output:
19
        patterns:
20
          - "\\b(confidential|internal[_-]?only|do[_-]?not[_-]?share)\\b"
21
        case_insensitive: true
22
23
  input:
24
    flows:
25
      - regex check input
26
27
  output:
28
    flows:
29
      - regex check output

Return Value

The detect_regex_pattern action returns a RegexDetectionResult with the following fields:

For example, if the input text "my SSN is 123-45-6789" is checked against a pattern \\d{3}-\\d{2}-\\d{4}, the result would be:

1
{
2
  "is_match": true,
3
  "text": "my SSN is 123-45-6789",
4
  "detections": ["\\d{3}-\\d{2}-\\d{4}"]
5
}

Regex Pattern Tips

When writing Python regex patterns, keep in mind:

1. Escape backslashes: In YAML, use double backslashes (\\) for regex escape sequences (e.g., \\b for word boundary, \\d for digit).

2. Word boundaries: Use \\b to match whole words only. For example, \\bpassword\\b matches “password” but not “passwords” or “mypassword”.

3. Character classes: Use [_-]? to match optional separators (e.g., api[_-]?key matches “apikey”, “api-key”, and “api_key”).

4. Common patterns:

   • SSN: \\d{3}-\\d{2}-\\d{4}
   • Credit card: \\d{4}[- ]?\\d{4}[- ]?\\d{4}[- ]?\\d{4}
   • Email: [a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}
   • Phone: \\(?\\d{3}\\)?[-.\\s]?\\d{3}[-.\\s]?\\d{4}

For more information about Python regex rules, please refer to the official documentation.

Regex Rail Behavior

When a regex rail has a regex pattern match, the following steps happen (depending on rails type):

Input and output rails

1. The action returns a RegexDetectionResult with is_match=True and the matched pattern(s) in detections.
2. The bot responds with: "I'm sorry, I can't respond to that." (using the standard bot refuse to respond message).
3. The flow is aborted.

Retrieval rails

1. Chunks that match any retrieval pattern are removed from the retrieval results. Only non-matching chunks are passed to the model as context.

NOTE: The matched pattern(s) are available in result["detections"] for logging and debugging. They are also logged at INFO level by the action.