Prompt Configuration for the NVIDIA NeMo Guardrails Library
This section describes how to customize prompts in the config.yml or prompts.yml file.
Task-Oriented Prompting
The interaction with the LLM is task-oriented: each LLM call performs a specific task. The core tasks in the guardrails process are:
For the complete list of tasks, see Task type.
The prompts Key
The prompts key allows you to customize the prompts used for various LLM tasks.
You can define prompts in the main config.yml file or in a separate prompts.yml file.
Basic Prompt Structure
For a complete list of available prompt attributes and tasks, refer to the YAML Schema Reference: Prompts Configuration.
Content-Based Prompts
The following example shows a simple prompt that uses the content attribute with Jinja2 templating:
Message-Based Prompts
For chat models, use the messages format:
Model-Specific Prompts
Override prompts for specific models:
Prompting Modes
Use the mode attribute to define multiple prompt versions for the same task and model. This enables prompt engineering experiments such as compact prompts for lower latency.
Configuration:
Prompt definition:
The mode in the prompt definition must match the prompting_mode in the top-level configuration. If no matching mode is found, the standard prompt is used.
Prompt Attributes Reference
Template Variables
Prompt templates use Jinja2 for variable substitution. Three types of variables are available:
System Variables
Prompt Variables
Register custom variables using the LLMRails.register_prompt_context() method:
If a function is provided, the value is computed for each rendering.
Context Variables
Flows in your guardrails configuration can define context variables. These variables are also available in prompt templates.
Filters
Filters modify variable content using the pipe symbol (|). The library provides these predefined filters:
Example:
Output Parsers
Use the output_parser attribute to parse LLM output. Available parsers:
Example Configurations
Self-Check Input
Self-Check Output
Fact Checking
Custom Tasks and Prompts
Define custom tasks beyond the built-in tasks by adding them to your prompts configuration:
Render custom task prompts in an action using LLMTaskManager:
Predefined Prompts
The library includes predefined prompts for these models:
openai/gpt-3.5-turbo-instructopenai/gpt-3.5-turboopenai/gpt-4databricks/dolly-v2-3bcohere/commandcohere/command-lightcohere/command-light-nightly
Predefined prompts are continuously evaluated and improved. Test and customize prompts for your specific use case before deploying to production.
Environment Variable
You can also load prompts from an external directory by setting:
The directory must contain .yml files with prompt definitions.