nemoguardrails.rails.llm.config | NVIDIA NeMo Guardrails Library Developer Guide

Module for the configuration of rails.

Module Contents

Classes

Name	Description
`ActionRails`	Configuration of action rails.
`CacheStatsConfig`	Configuration for cache statistics tracking and logging.
`CoreConfig`	Settings for core internal mechanics.
`DialogRails`	Configuration of topical rails.
`Document`	Configuration for documents that should be used for question answering.
`EmbeddingSearchProvider`	Configuration of a embedding search provider.
`EmbeddingsCacheConfig`	Configuration for the caching embeddings.
`InputRails`	Configuration of input rails.
`Instruction`	Configuration for instructions in natural language that should be passed to the LLM.
`KnowledgeBaseConfig`	-
`LogAdapterConfig`	-
`MessageTemplate`	Template for a message structure.
`MetricsConfig`	OpenTelemetry Metrics Configuration.
`Model`	Configuration of a model used by the rails engine.
`ModelCacheConfig`	Configuration for model caching.
`OutputRails`	Configuration of output rails.
`OutputRailsStreamingConfig`	Configuration for managing streaming output of LLM tokens.
`Rails`	Configuration of specific rails.
`RailsConfig`	Configuration object for the models and the rails.
`RailsConfigData`	-
`RetrievalRails`	Configuration of retrieval rails.
`SingleCallConfig`	Configuration for the single LLM call option for topical rails.
`SpanFormat`	-
`TaskPrompt`	Configuration for prompts that will be used for a specific task.
`ToolInputRails`	Configuration of tool input rails.
`ToolOutputRails`	Configuration of tool output rails.
`TracingConfig`	-
`UserMessagesConfig`	Configuration for how the user messages are interpreted.
`_RailsConfigDataBase`	Configuration data for specific rails that are supported out-of-the-box.

Functions

Name	Description
`__dir__`	-
`__getattr__`	-
`_generate_rails_flows`	Generates flow definitions from the list of flows.
`_get_flow_model`	Helper to return a model name from a flow definition
`_get_flow_name`	Helper to return a model name from a flow definition
`_get_rails_flows`	Extracts the list of flows from the raw_config dictionary.
`_has_input_output_config_rails`	Checks if the raw configuration has input/output rails configured.
`_join_config`	Helper to join two configuration.
`_join_dict`	Joins two dictionaries recursively.
`_join_rails_configs`	Helper to join two rails configuration.
`_load_imported_paths`	Load recursively all the imported path in the specified raw_config.
`_load_path`	Load a configuration object from the specified path.
`_parse_colang_files_recursively`	Helper function to parse all the Colang files.
`_unique_list_concat`	Concatenates two lists ensuring all elements are unique.
`_validate_rail_prompts`	-
`_validate_self_check_rail_prompts`	Ensure every enabled self-check rail has a prompt for its resolved task.
`merge_two_dicts`	Merges the fields of two dictionaries recursively.

Data

JAILBREAK_FLOW_HEURISTICS

JAILBREAK_FLOW_MODEL

guardrails_stdlib_path

log

standard_library_path

API

class nemoguardrails.rails.llm.config.ActionRails()

Bases: BaseModel

Configuration of action rails.

Action rails control various options related to the execution of actions. Currently, only

In the future multiple options will be added, e.g., what input validation should be performed per action, output validation, throttling, disabling, etc.

instant_actions

Optional[List[str]]

class nemoguardrails.rails.llm.config.CacheStatsConfig()

Bases: BaseModel

Configuration for cache statistics tracking and logging.

enabled

bool

log_interval

Optional[float]

class nemoguardrails.rails.llm.config.CoreConfig()

Bases: BaseModel

Settings for core internal mechanics.

embedding_search_provider

EmbeddingSearchProvider

class nemoguardrails.rails.llm.config.DialogRails()

Bases: BaseModel

Configuration of topical rails.

single_call

SingleCallConfig

user_messages

UserMessagesConfig

class nemoguardrails.rails.llm.config.Document()

Bases: BaseModel

Configuration for documents that should be used for question answering.

content

str

format

str

class nemoguardrails.rails.llm.config.EmbeddingSearchProvider()

Bases: BaseModel

Configuration of a embedding search provider.

cache

EmbeddingsCacheConfig = Field(default_factory=EmbeddingsCacheConfig)

name

str

parameters

Dict[str, Any] = Field(default_factory=dict)

class nemoguardrails.rails.llm.config.EmbeddingsCacheConfig()

Bases: BaseModel

Configuration for the caching embeddings.

enabled

bool

key_generator

str

store

str

store_config

Dict[str, Any]

nemoguardrails.rails.llm.config.EmbeddingsCacheConfig.to_dict()

class nemoguardrails.rails.llm.config.InputRails()

Bases: BaseModel

Configuration of input rails.

flows

List[str]

parallel

Optional[bool]

speculative_generation

Optional[bool]

class nemoguardrails.rails.llm.config.Instruction()

Bases: BaseModel

Configuration for instructions in natural language that should be passed to the LLM.

content

str

type

str

class nemoguardrails.rails.llm.config.KnowledgeBaseConfig()

Bases: BaseModel

embedding_search_provider

EmbeddingSearchProvider

folder

str

class nemoguardrails.rails.llm.config.LogAdapterConfig()

Bases: BaseModel

model_config

= ConfigDict(extra='allow')

name

str

class nemoguardrails.rails.llm.config.MessageTemplate()

Bases: BaseModel

Template for a message structure.

content

str = Field(description='The content of the message.')

type

str

class nemoguardrails.rails.llm.config.MetricsConfig()

Bases: BaseModel

OpenTelemetry Metrics Configuration. Configures and enables Metrics independent of OTEL Traces.

enabled

bool

class nemoguardrails.rails.llm.config.Model()

Bases: BaseModel

Configuration of a model used by the rails engine.

Typically, the main model is configured e.g.: { “type”: “main”, “engine”: “openai”, “model”: “gpt-3.5-turbo-instruct” }

api_key_env_var

Optional[str]

cache

Optional[ModelCacheConfig]

engine

str

mode

Literal['chat', 'text']

model

Optional[str]

parameters

Dict[str, Any] = Field(default_factory=dict)

type

str

nemoguardrails.rails.llm.config.Model.model_must_be_none_empty() -> nemoguardrails.rails.llm.config.Model

Validate that a model name is present either directly or in parameters.

nemoguardrails.rails.llm.config.Model.set_and_validate_model(
    data: typing.Any
) -> typing.Any

classmethod

class nemoguardrails.rails.llm.config.ModelCacheConfig()

Bases: BaseModel

Configuration for model caching.

enabled

bool

maxsize

int

stats

CacheStatsConfig

class nemoguardrails.rails.llm.config.OutputRails()

Bases: BaseModel

Configuration of output rails.

flows

List[str]

parallel

Optional[bool]

streaming

OutputRailsStreamingConfig

class nemoguardrails.rails.llm.config.OutputRailsStreamingConfig()

Bases: BaseModel

Configuration for managing streaming output of LLM tokens.

chunk_size

int

context_size

int

enabled

bool

model_config

= ConfigDict(extra='allow')

stream_first

bool

class nemoguardrails.rails.llm.config.Rails()

Bases: BaseModel

Configuration of specific rails.

actions

ActionRails

config

RailsConfigData

dialog

DialogRails

input

InputRails

output

OutputRails

retrieval

RetrievalRails

tool_input

ToolInputRails

tool_output

ToolOutputRails

class nemoguardrails.rails.llm.config.RailsConfig()

Bases: BaseModel

Configuration object for the models and the rails.

TODO: add typed config for user_messages, bot_messages, and flows.

actions_server_url

Optional[str]

bot_messages

Dict[str, List[str]]

colang_version

str

config_path

Optional[str]

core

CoreConfig

custom_data

Dict

docs

Optional[List[Document]]

enable_multi_step_generation

Optional[bool]

enable_rails_exceptions

bool

event_source_uid

str

flows

List[Union[Dict, Any]]

import_paths

Optional[List[str]]

imported_paths

Optional[Dict[str, str]]

instructions

Optional[List[Instruction]]

knowledge_base

KnowledgeBaseConfig

lowest_temperature

Optional[float]

metrics

MetricsConfig

models

List[Model]

passthrough

Optional[bool]

prompting_mode

Optional[str]

prompts

Optional[List[TaskPrompt]]

rails

Rails

raw_llm_call_action

Optional[str]

sample_conversation

Optional[str]

streaming

bool

tracing

TracingConfig

user_messages

Dict[str, List[str]]

nemoguardrails.rails.llm.config.RailsConfig.__add__(
    other
)

Adds two RailsConfig objects.

nemoguardrails.rails.llm.config.RailsConfig.check_jailbreak_detection_config(
    values
)

classmethod

Validate jailbreak detection configuration against enabled flows.

nemoguardrails.rails.llm.config.RailsConfig.check_model_exists_for_input_rails(
    values
)

classmethod

Make sure we have a model for each input rail where one is provided using $model=<model_type>

nemoguardrails.rails.llm.config.RailsConfig.check_model_exists_for_output_rails(
    values
)

classmethod

Make sure we have a model for each output rail where one is provided using $model=<model_type>

nemoguardrails.rails.llm.config.RailsConfig.check_output_parser_exists(
    values
)

classmethod

nemoguardrails.rails.llm.config.RailsConfig.check_prompt_exist_for_self_check_rails(
    values
)

classmethod

nemoguardrails.rails.llm.config.RailsConfig.check_streaming_can_apply_output_rewrites(
    values
)

classmethod

Reject configs whose output rails transform response with stream_first == True

nemoguardrails.rails.llm.config.RailsConfig.fill_in_default_values_for_v2_x(
    values
)

classmethod

nemoguardrails.rails.llm.config.RailsConfig.from_content(
    colang_content: typing.Optional[str] = None,
    yaml_content: typing.Optional[str] = None,
    config: typing.Optional[dict] = None
)

classmethod

Loads a configuration from the provided colang/YAML content/config dict.

nemoguardrails.rails.llm.config.RailsConfig.from_path(
    config_path: str
)

classmethod

Loads a configuration from a given path.

Supports loading a from a single file, or from a directory.

nemoguardrails.rails.llm.config.RailsConfig.parse_object(
    obj
)

classmethod

Parses a configuration object from a given dictionary.

nemoguardrails.rails.llm.config.RailsConfig.validate_models_api_key_env_var(
    models
)

classmethod

Model API Key Env var must be set to make LLM calls

class nemoguardrails.rails.llm.config.RailsConfigData()

Bases: _RailsConfigDataBase

nemoguardrails.rails.llm.config.RailsConfigData.__getattr__(
    name: str
) -> typing.Any

class nemoguardrails.rails.llm.config.RetrievalRails()

Bases: BaseModel

Configuration of retrieval rails.

flows

List[str]

class nemoguardrails.rails.llm.config.SingleCallConfig()

Bases: BaseModel

Configuration for the single LLM call option for topical rails.

enabled

bool = False

fallback_to_multiple_calls

bool

class nemoguardrails.rails.llm.config.SpanFormat

Bases: enum.Enum

legacy

= 'legacy'

opentelemetry

= 'opentelemetry'

class nemoguardrails.rails.llm.config.TaskPrompt()

Bases: BaseModel

Configuration for prompts that will be used for a specific task.

content

Optional[str]

max_length

Optional[int]

max_tokens

Optional[int]

messages

Optional[List[Union[MessageTemplate, str]]]

mode

Optional[str]

models

Optional[List[str]]

output_parser

Optional[str]

stop

Optional[List[str]]

task

str

nemoguardrails.rails.llm.config.TaskPrompt.check_fields(
    values
)

classmethod

class nemoguardrails.rails.llm.config.ToolInputRails()

Bases: BaseModel

Configuration of tool input rails.

Tool input rails are applied to tool results before they are processed. They can validate, filter, or transform tool outputs for security and safety.

flows

List[str]

parallel

Optional[bool]

class nemoguardrails.rails.llm.config.ToolOutputRails()

Bases: BaseModel

Configuration of tool output rails.

Tool output rails are applied to tool calls before they are executed. They can validate tool names, parameters, and context to ensure safe tool usage.

flows

List[str]

parallel

Optional[bool]

class nemoguardrails.rails.llm.config.TracingConfig()

Bases: BaseModel

adapters

List[LogAdapterConfig]

enable_content_capture

bool

enabled

bool = False

span_format

str

class nemoguardrails.rails.llm.config.UserMessagesConfig()

Bases: BaseModel

Configuration for how the user messages are interpreted.

embeddings_only

bool

embeddings_only_fallback_intent

Optional[str]

embeddings_only_similarity_threshold

Optional[float]

class nemoguardrails.rails.llm.config._RailsConfigDataBase()

Bases: BaseModel

Configuration data for specific rails that are supported out-of-the-box.

nemoguardrails.rails.llm.config.__dir__()

nemoguardrails.rails.llm.config.__getattr__(
    name: str
)

nemoguardrails.rails.llm.config._generate_rails_flows(
    flows
)

Generates flow definitions from the list of flows. Args: flows (dict): The dictionary of flows. Returns: str: The flow definitions.

nemoguardrails.rails.llm.config._get_flow_model(
    flow_text
) -> typing.Optional[str]

Helper to return a model name from a flow definition

nemoguardrails.rails.llm.config._get_flow_name(
    flow_text
) -> typing.Optional[str]

Helper to return a model name from a flow definition

nemoguardrails.rails.llm.config._get_rails_flows(
    raw_config
)

Extracts the list of flows from the raw_config dictionary.

Parameters:

raw_config

dict

The raw configuration dictionary.

Returns:

The list of flows.

nemoguardrails.rails.llm.config._has_input_output_config_rails(
    raw_config
)

Checks if the raw configuration has input/output rails configured.

nemoguardrails.rails.llm.config._join_config(
    dest_config: dict,
    additional_config: dict
)

Helper to join two configuration.

nemoguardrails.rails.llm.config._join_dict(
    dict1,
    dict2
)

Joins two dictionaries recursively.

If values are dictionaries, it applies _join_dict recursively.
If values are lists, it concatenates them, ensuring unique elements.
For other types, values from dict2 overwrite dict1.

nemoguardrails.rails.llm.config._join_rails_configs(
    base_rails_config: nemoguardrails.rails.llm.config.RailsConfig,
    updated_rails_config: nemoguardrails.rails.llm.config.RailsConfig
)

Helper to join two rails configuration.

nemoguardrails.rails.llm.config._load_imported_paths(
    raw_config: dict,
    colang_files: typing.List[typing.Tuple[str, str]]
)

Load recursively all the imported path in the specified raw_config.

Parameters:

raw_config

dict

The starting raw configuration (i.e., a dict)

colang_files

List[Tuple[str, str]]

The current set of colang files which will be extended as new configurations are loaded.

nemoguardrails.rails.llm.config._load_path(
    config_path: str
) -> typing.Tuple[dict, typing.List[typing.Tuple[str, str]]]

Load a configuration object from the specified path.

Parameters:

config_path

str

The path from which to load.

Returns: Tuple[dict, List[Tuple[str, str]]]

(raw_config, colang_files) The raw config object and the list of colang files.

nemoguardrails.rails.llm.config._parse_colang_files_recursively(
    raw_config: dict,
    colang_files: typing.List[typing.Tuple[str, str]],
    parsed_colang_files: typing.List[dict]
)

Helper function to parse all the Colang files.

If there are imports, they will be imported recursively

nemoguardrails.rails.llm.config._unique_list_concat(
    list1,
    list2
)

Concatenates two lists ensuring all elements are unique. Handles unhashable types like dictionaries.

nemoguardrails.rails.llm.config._validate_rail_prompts(
    rails: list[str],
    prompts: list[typing.Any],
    validation_rail: str
) -> None

nemoguardrails.rails.llm.config._validate_self_check_rail_prompts(
    rails: list[str],
    prompts: list[typing.Any],
    validation_rail: str,
    default_task: str
) -> None

Ensure every enabled self-check rail has a prompt for its resolved task.

A self-check rail may select a custom task via $variant=...; the task falls back to default_task when omitted. Custom prompts are scoped under the default task name.

nemoguardrails.rails.llm.config.merge_two_dicts(
    dict_1: dict,
    dict_2: dict,
    ignore_keys: typing.Set[str]
) -> None

Merges the fields of two dictionaries recursively.

nemoguardrails.rails.llm.config.JAILBREAK_FLOW_HEURISTICS = 'jailbreak detection heuristics'

nemoguardrails.rails.llm.config.JAILBREAK_FLOW_MODEL = 'jailbreak detection model'

nemoguardrails.rails.llm.config.MODEL_PREFIX = '$model='

nemoguardrails.rails.llm.config.__all__ = sorted({name for name in (globals()) if not name.startswith('_')} | set(config_e...

nemoguardrails.rails.llm.config._default_config = yaml.safe_load(_fc)

nemoguardrails.rails.llm.config._default_config_v2 = yaml.safe_load(_fc)

nemoguardrails.rails.llm.config.colang_path_dirs = [(_path.strip()) for _path in (os.environ.get('COLANGPATH', '').split(os.pathsep...

nemoguardrails.rails.llm.config.guardrails_stdlib_path = os.path.normpath(os.path.join(os.path.dirname(__file__), '..', '..', '..'))

nemoguardrails.rails.llm.config.log = logging.getLogger(__name__)

nemoguardrails.rails.llm.config.standard_library_path = os.path.normpath(os.path.join(os.path.dirname(__file__), '..', '..', 'colang', '...