Retrieval-Augmented Generation

This guide shows how to apply a guardrails configuration in a RAG scenario. This guide builds on the previous guide, developing further the demo ABC Bot.

Prerequisites

1. Install the openai package:

$
pip install openai

1. Set the OPENAI_API_KEY environment variable:

$
export OPENAI_API_KEY=$OPENAI_API_KEY    # Replace with your own key

1. If you’re running this inside a notebook, patch the AsyncIO loop.

1
import nest_asyncio
2
3
nest_asyncio.apply()

Usage

There are two modes in which you can use a guardrails configuration in conjunction with RAG:

1. Relevant Chunks: perform the retrieval yourself and pass the relevant chunks directly to the generate method.
2. Knowledge Base: configure a knowledge base directly into the guardrails configuration and let NeMo Guardrails manage the retrieval part.

Relevant Chunks

In the previous guide, the message “How many free vacation days do I have per year” yields a general response:

1
from nemoguardrails import RailsConfig, LLMRails
2
3
config = RailsConfig.from_path("./config")
4
rails = LLMRails(config)
5
6
response = rails.generate(messages=[{
7
    "role": "user",
8
    "content": "How many vacation days do I have per year?"
9
}])
10
print(response["content"])

Full-time employees are eligible for up to two weeks of paid vacation time per year. Part-time employees receive a prorated amount based on their hours worked. Please refer to the employee handbook for more information.

ABC company’s Employee Handbook contains the following information:

1
Employees are eligible for the following time off:
2
3
* Vacation: 20 days per year, accrued monthly.
4
* Sick leave: 15 days per year, accrued monthly.
5
* Personal days: 5 days per year, accrued monthly.
6
* Paid holidays: New Year's Day, Memorial Day, Independence Day, Thanksgiving Day, Christmas Day.
7
* Bereavement leave: 3 days paid leave for immediate family members, 1 day for non-immediate family members.

You can pass this information directly to guardrails when making a generate call:

1
response = rails.generate(messages=[{
2
    "role": "context",
3
    "content": {
4
        "relevant_chunks": """
5
            Employees are eligible for the following time off:
6
              * Vacation: 20 days per year, accrued monthly.
7
              * Sick leave: 15 days per year, accrued monthly.
8
              * Personal days: 5 days per year, accrued monthly.
9
              * Paid holidays: New Year's Day, Memorial Day, Independence Day, Thanksgiving Day, Christmas Day.
10
              * Bereavement leave: 3 days paid leave for immediate family members, 1 day for non-immediate family members. """
11
    }
12
},{
13
    "role": "user",
14
    "content": "How many vacation days do I have per year?"
15
}])
16
print(response["content"])

Eligible employees receive 20 days of paid vacation time per year, which accrues monthly. You can find more information about this in the employee handbook.

As expected, the response contains the correct answer.

Knowledge Base

There are three ways you can configure a knowledge base directly into a guardrails configuration:

1. Using the kb folder.
2. Using a custom retrieve_relevant_chunks action.
3. Using a custom EmbeddingSearchProvider.

For option 1, you can add a knowledge base directly into your guardrails configuration by creating a kb folder inside the config folder and adding documents there. Currently, only the Markdown format is supported. For a quick example, check out the complete implementation of the ABC Bot.

Options 2 and 3 represent advanced use cases beyond the scope of this topic.

Wrapping Up

This guide introduced how a guardrails configuration can be used in the context of a RAG setup.

Next

To continue learning about NeMo Guardrails, check out:

1. Guardrails Catalog.
2. Configuration Reference.