Skip to navigation

The NeMo Guardrails library.

Subpackages

Submodules

Package Contents

Classes

NameDescription
ChatMessage-
GuardrailsTop-level interface for NeMo Guardrails functionality.
LLMFrameworkProtocol for pluggable LLM framework backends.
LLMModelProtocol that all LLM backends must implement.
LLMRailsRails based on a given configuration.
LLMRailsRemovalWarningA feature that is removed together with the LLMRails engine.
LLMResponse-
LLMResponseChunk-
RailsConfigConfiguration object for the models and the rails.
Role-
ToolCall-
ToolCallChunkA structured stream event containing completed tool calls, not argument fragments.
ToolCallDelta-
ToolCallDeltaChunk-
ToolCallFunction-
ToolCallFunctionDelta-
UsageInfo-

Functions

NameDescription
get_default_framework-
register_frameworkRegister an LLMFramework instance under name.
register_provider-
set_default_framework-

Data

FinishReason

__version__

_use_guardrails_wrapper

API

class nemoguardrails.ChatMessage(
content: typing.Optional[typing.Union[str, typing.List[typing.Dict[str, typing.Any]]]] = None,
tool_calls: typing.Optional[typing.List[nemoguardrails.types.ToolCall]] = None,
tool_call_id: typing.Optional[str] = None,
name: typing.Optional[str] = None,
provider_metadata: typing.Dict[str, typing.Any] = dict()
)
Dataclass
content
Optional[Union[str, List[Dict[str, Any]]]] = None
name
Optional[str] = None
provider_metadata
Dict[str, Any] = field(default_factory=dict)
role
Role
tool_call_id
Optional[str] = None
tool_calls
Optional[List[ToolCall]] = None
nemoguardrails.ChatMessage.from_assistant(
content: str,
kwargs = {}
classmethod
nemoguardrails.ChatMessage.from_dict(
d: typing.Dict[str, typing.Any]
classmethod

Create a ChatMessage from a dict.

Accepts both the canonical nested tool call format ({"function": {"name": ..., "arguments": ...}}) and the legacy flat format ({"name": ..., "args": ...}). JSON string arguments are parsed automatically. Role aliases like “bot”, “human”, and “developer” are mapped to canonical Role values. Unknown keys are captured into provider_metadata.

nemoguardrails.ChatMessage.from_system(
content: str,
kwargs = {}
classmethod
nemoguardrails.ChatMessage.from_tool(
content: str,
tool_call_id: str,
kwargs = {}
classmethod
nemoguardrails.ChatMessage.from_user(
content: str,
kwargs = {}
classmethod
nemoguardrails.ChatMessage.to_dict() -> typing.Dict[str, typing.Any]
class nemoguardrails.Guardrails(
llm: typing.Optional[nemoguardrails.types.LLMModel] = None,
verbose: bool = False,
use_iorails: bool = True,
require_iorails: bool = False
)

Bases: BaseGuardrails

Top-level interface for NeMo Guardrails functionality.

_rails_engine
= IORails(config)
_started
bool

Read IORails readiness from its owner; retain legacy startup state.

events_history_cache
dict

Per-session events history cache.

Used by the server to persist conversation state across requests. For LLMRails this is stored by reference; assigning replaces the dict object, not its contents.

IORails is stateless, return empty cache and drop cache-store writes

explain_info
Optional[ExplainInfo]

Deprecated. Use explain() instead.

Direct access can return None for an uninitialized accumulator; explain() guarantees a non-None ExplainInfo. Only supported for LLMRails.

llm
Optional[LLMModel]

The main LLM in use. Only supported for LLMRails. Read-only; use update_llm() to replace it.

passthrough_fn
Optional[Callable]

The optional passthrough function that bypasses LLM generation.

Only supported for LLMRails. When set, the rails pipeline calls this function instead of the main LLM for generating responses.

rails_engine
IORails | LLMRails

Get immutable LLMRails object

runtime
Runtime

The Colang runtime backing the rails engine. Only supported for LLMRails.

use_iorails_engine
bool
nemoguardrails.Guardrails.__aenter__()
async

Async context manager entry.

nemoguardrails.Guardrails.__aexit__(
exc_type,
exc_val,
exc_tb
)
async

Async context manager exit.

nemoguardrails.Guardrails.__getstate__()

Pickle support: preserve config, verbose, and use_iorails so the rebuilt instance lands on the same engine. The llm is dropped (matches LLMRails).

nemoguardrails.Guardrails.__setstate__(
state
)

Pickle support: rebuild from config + verbose + use_iorails. Older pickles missing these keys default to False/True respectively for backwards compatibility.

nemoguardrails.Guardrails._ensure_started() -> None
async

Lazy initialization: call startup() on first use if not already started.

nemoguardrails.Guardrails.check(
messages: typing.List[dict],
rail_types: typing.Optional[typing.List[nemoguardrails.rails.llm.options.RailType]] = None,
tools: typing.Optional[typing.List[dict]] = None

Synchronous version of check_async. Supported by both LLMRails and IORails; tool rail types and tools by IORails only.

nemoguardrails.Guardrails.check_async(
messages: typing.List[dict],
rail_types: typing.Optional[typing.List[nemoguardrails.rails.llm.options.RailType]] = None,
tools: typing.Optional[typing.List[dict]] = None
async

Run rails on messages based on their content (asynchronous). Supported by both LLMRails and IORails; tool rail types and tools by IORails only.

nemoguardrails.Guardrails.explain() -> nemoguardrails.logging.explain.ExplainInfo

Get the latest ExplainInfo object for debugging. Only supported for LLMRails

nemoguardrails.Guardrails.generate(
prompt: str | None = None,
options: typing.Optional[typing.Union[dict, nemoguardrails.rails.llm.options.GenerationOptions]] = None,
kwargs = {}
) -> typing.Union[str, dict, nemoguardrails.rails.llm.options.GenerationResponse, typing.Tuple[dict, dict]]

Generate an LLM response synchronously with guardrails applied. Supported in both IORails and LLMRails

nemoguardrails.Guardrails.generate_async(
prompt: str | None = None,
options: typing.Optional[typing.Union[dict, nemoguardrails.rails.llm.options.GenerationOptions]] = None,
kwargs = {}
) -> str | dict | nemoguardrails.rails.llm.options.GenerationResponse | tuple[dict, dict]
async

Generate an LLM response asynchronously with guardrails applied. Supported by both LLMRails and IORails

nemoguardrails.Guardrails.generate_events(
events: typing.List[dict]
) -> typing.List[dict]

Synchronous version of generate_events_async. Only supported for LLMRails.

nemoguardrails.Guardrails.generate_events_async(
events: typing.List[dict]
) -> typing.List[dict]
async

Generate the next events based on the provided history. Only supported for LLMRails.

nemoguardrails.Guardrails.process_events(
events: typing.List[dict],
state: typing.Union[typing.Optional[dict], nemoguardrails.colang.v2_x.runtime.flows.State] = None,
blocking: bool = False
) -> typing.Tuple[typing.List[dict], typing.Union[dict, nemoguardrails.colang.v2_x.runtime.flows.State]]

Synchronous version of process_events_async. Only supported for LLMRails.

nemoguardrails.Guardrails.process_events_async(
events: typing.List[dict],
state: typing.Union[typing.Optional[dict], nemoguardrails.colang.v2_x.runtime.flows.State] = None,
blocking: bool = False
) -> typing.Tuple[typing.List[dict], typing.Union[dict, nemoguardrails.colang.v2_x.runtime.flows.State]]
async

Process a sequence of events in a given state. Only supported for LLMRails.

nemoguardrails.Guardrails.register_action(
action: typing.Callable,
name: typing.Optional[str] = None
) -> typing_extensions.Self

Register a custom action for the rails configuration. Only supported for LLMRails. Returns self so calls can be chained.

nemoguardrails.Guardrails.register_action_param(
name: str,
value: typing.Any
) -> typing_extensions.Self

Register a custom action parameter. Only supported for LLMRails. Returns self so calls can be chained.

nemoguardrails.Guardrails.register_embedding_provider(
name: typing.Optional[str] = None
) -> typing_extensions.Self

Register a custom embedding provider. Only supported for LLMRails. Returns self so calls can be chained.

nemoguardrails.Guardrails.register_embedding_search_provider(
name: str
) -> typing_extensions.Self

Register a new embedding search provider. Only supported for LLMRails. Returns self so calls can be chained.

nemoguardrails.Guardrails.register_filter(
filter_fn: typing.Callable,
name: typing.Optional[str] = None
) -> typing_extensions.Self

Register a custom filter for the rails configuration. Only supported for LLMRails. Returns self so calls can be chained.

nemoguardrails.Guardrails.register_output_parser(
output_parser: typing.Callable,
name: str
) -> typing_extensions.Self

Register a custom output parser for the rails configuration. Only supported for LLMRails. Returns self so calls can be chained.

nemoguardrails.Guardrails.register_prompt_context(
name: str,
value_or_fn: typing.Any
) -> typing_extensions.Self

Register a value to be included in the prompt context. Only supported for LLMRails. Returns self so calls can be chained.

nemoguardrails.Guardrails.shutdown() -> None
async

Lifecycle method to stop the rails engine.

Idempotent after successful cleanup. Failed or cancelled IORails startup and shutdown may retain resources; call again to retry their cleanup.

nemoguardrails.Guardrails.startup() -> None
async

Lifecycle method to start the rails engine.

Idempotent: safe to call multiple times. Also called automatically on first generate_async() if not called explicitly, so callers are not required to manage the lifecycle.

The non-streaming admission queue is owned by IORails and is started/stopped as part of IORails.start() / stop().

nemoguardrails.Guardrails.stream_async(
prompt: str | None = None,
kwargs = {}

Generate an LLM response asynchronously with streaming support.

nemoguardrails.Guardrails.update_llm(
) -> None

Replace the main LLM with a new one. Only supported for LLMRails, since IORails doesn’t take LLM as argument

class nemoguardrails.LLMFramework()
Protocol

Protocol for pluggable LLM framework backends.

Each framework (LangChain, LiteLLM, etc.) implements this protocol to provide a factory for creating LLMModel instances and managing its own set of providers.

model_kwargs carries all provider-specific configuration. Framework implementations extract what they need (e.g. LangChain pops mode to choose between chat and text completion models).

nemoguardrails.LLMFramework.create_model(
model_name: str,
provider_name: str,
model_kwargs: typing.Optional[typing.Dict[str, typing.Any]] = None
nemoguardrails.LLMFramework.get_provider_names() -> typing.List[str]
nemoguardrails.LLMFramework.register_provider(
name: str,
provider_cls: typing.Any
) -> None
nemoguardrails.LLMFramework.reset() -> None
async

Release all framework-owned resources and clear all registered state.

Implementations should close any pooled connections, clear registered providers, and return the framework to its initial state. Callers can continue using the instance after reset; new resources will be created on demand.

class nemoguardrails.LLMModel()
Protocol

Protocol that all LLM backends must implement.

Adapters wrap provider-specific SDKs (LangChain, LiteLLM, OpenAI, etc.) behind this interface so the core pipeline remains framework-agnostic.

prompt accepts either a plain string or a list of ChatMessage objects. Adapters convert ChatMessage to whatever their SDK expects. **kwargs are forwarded to the underlying SDK (e.g. temperature, max_tokens).

model_name
str
provider_name
Optional[str]
provider_url
Optional[str]
nemoguardrails.LLMModel.generate_async(
prompt: typing.Union[str, typing.List[nemoguardrails.types.ChatMessage]],
stop: typing.Optional[typing.List[str]] = None,
kwargs = {}
async
nemoguardrails.LLMModel.stream_async(
prompt: typing.Union[str, typing.List[nemoguardrails.types.ChatMessage]],
stop: typing.Optional[typing.List[str]] = None,
kwargs = {}
) -> typing.AsyncIterator[nemoguardrails.types.LLMResponseChunk]

Implementations must be async generator functions (use yield).

class nemoguardrails.LLMRails(
llm: typing.Optional[nemoguardrails.types.LLMModel] = None,
verbose: bool = False
)

Bases: BaseGuardrails

Rails based on a given configuration.

_default_embedding_engine
= 'FastEmbed'
_default_embedding_model
= 'all-MiniLM-L6-v2'
_default_embedding_params
= {}
_embedding_search_providers
= {}
_llm_generation_actions
_log_adapters
= create_log_adapters(config.tracing)
events_history_cache
= {}
llm
Optional[LLMModel]
passthrough_fn

The optional passthrough function that bypasses LLM generation.

When set, the rails pipeline calls this function instead of the main LLM for generating responses. LLMGenerationActions is private, expose only passthrough_fn as a public API

runtime
Runtime
nemoguardrails.LLMRails.__getstate__()
nemoguardrails.LLMRails.__setstate__(
state
)
nemoguardrails.LLMRails._create_model_cache(
model

Create cache instance for a model based on its configuration.

Parameters:

model

The model configuration object

Returns: LFUCache

The cache instance

nemoguardrails.LLMRails._ensure_explain_info() -> nemoguardrails.logging.explain.ExplainInfo
staticmethod

Ensure that the ExplainInfo variable is present in the current context

Returns: ExplainInfo

A ExplainInfo class containing the llm calls’ statistics

nemoguardrails.LLMRails._get_embeddings_search_provider_instance(
esp_config: typing.Optional[nemoguardrails.rails.llm.config.EmbeddingSearchProvider] = None
nemoguardrails.LLMRails._get_events_for_messages(
messages: typing.List[dict],
state: typing.Any
)

Return the list of events corresponding to the provided messages.

Tries to find a prefix of messages for which we have already a list of events in the cache. For the rest, they are converted as is.

The reason this cache exists is that we want to benefit from events generated in previous turns, which can’t be computed again because it would be expensive (e.g., involving multiple LLM calls).

When an explicit state object will be added, this mechanism can be removed.

Parameters:

messages
List[dict]

The list of messages.

Returns:

A list of events.

nemoguardrails.LLMRails._init_kb()
async

Initializes the knowledge base.

nemoguardrails.LLMRails._init_llms()

Initializes the right LLM engines based on the configuration. There can be multiple LLM engines and types that can be specified in the config. The main LLM engine is the one that will be used for all the core guardrails generations. Other LLM engines can be specified for use in specific actions.

The reason we provide an option for decoupling the main LLM engine from the action LLM is to allow for flexibility in using specialized LLM engines for specific actions.

Raises:

  • ModelInitializationError: If any model initialization fails
nemoguardrails.LLMRails._initialize_model_caches() -> None

Initialize caches for configured models.

nemoguardrails.LLMRails._prepare_model_kwargs(
model_config
)

Prepare kwargs for model initialization, including API key from environment variable.

Parameters:

model_config

The model configuration object

Returns:

The prepared kwargs for model initialization

nemoguardrails.LLMRails._resolve_streaming_output_rail_actions() -> typing.List[typing.Tuple[str, typing.Tuple[str, typing.Any]]]

Pair each configured output rail flow, in order, with the action streaming calls on every chunk.

nemoguardrails.LLMRails._run_output_rails_in_streaming(
streaming_handler: typing.AsyncIterator[typing.Union[str, dict, nemoguardrails.types.ToolCallChunk, nemoguardrails.types.ToolCallDeltaChunk]],
output_rail_actions: typing.List[typing.Tuple[str, typing.Tuple[str, typing.Any]]],
prompt: typing.Optional[str] = None,
messages: typing.Optional[typing.List[dict]] = None,
stream_first: typing.Optional[bool] = None
) -> typing.AsyncIterator[str]
async
  1. Buffers tokens from ‘streaming_handler’ via BufferStrategy.
  2. Runs sequential (parallel for colang 2.0 in future) flows for each chunk.
  3. Yields the chunk if not blocked, or STOP if blocked.
nemoguardrails.LLMRails._validate_config()

Runs additional validation checks on the config.

nemoguardrails.LLMRails._validate_public_state(
state: typing.Optional[typing.Union[dict, nemoguardrails.colang.v2_x.runtime.flows.State]]
) -> None

Validate public dict state passed through generate/generate_async.

nemoguardrails.LLMRails._validate_streaming_with_output_rails() -> None
nemoguardrails.LLMRails.check(
messages: typing.List[dict],
rail_types: typing.Optional[typing.List[nemoguardrails.rails.llm.options.RailType]] = None,
tools: typing.Optional[typing.List[dict]] = None

Run rails on messages based on their content (synchronous).

This is a synchronous wrapper around check_async().

Parameters:

messages
List[dict]

List of message dicts with ‘role’ and ‘content’ fields.

rail_types
Optional[List[RailType]]Defaults to None

Optional[List[ RailType ]] — Optional list of rail types to run. See check_async() for details.

tools
Optional[List[dict]]Defaults to None

Not supported. See check_async() for details.

Returns: RailsResult

RailsResult containing status, content, and optional blocking rail name.

Raises:

  • RailTypeNotSupportedError: If tools is given, or a requested rail type is tool_call or tool_result, which only the IORails engine runs.
  • RailTypeNotConfiguredError: If a requested rail type has no configured flows.
nemoguardrails.LLMRails.check_async(
messages: typing.List[dict],
rail_types: typing.Optional[typing.List[nemoguardrails.rails.llm.options.RailType]] = None,
tools: typing.Optional[typing.List[dict]] = None
async

Run rails on messages based on their content (asynchronous).

When rail_types is not provided, automatically determines which rails to run based on message roles:

  • Only user messages: runs input rails
  • Only assistant messages: runs output rails
  • Both user and assistant messages: runs both input and output rails
  • No user/assistant messages: logs warning and returns passing result

When rail_types is provided, runs exactly the specified rail types, skipping the auto-detection logic.

Parameters:

messages
List[dict]

List of message dicts with ‘role’ and ‘content’ fields. Messages can contain any roles, but only user/assistant roles determine which rails execute when rail_types is not provided.

rail_types
Optional[List[RailType]]Defaults to None

Optional[List[ RailType ]] — Optional list of rail types to run, e.g. [RailType.INPUT] or [RailType.OUTPUT]. When provided, overrides automatic detection.

tools
Optional[List[dict]]Defaults to None

Not supported. Only a tool-call check reads tools, and only the IORails engine runs one.

Returns: RailsResult

RailsResult containing:

Raises:

  • RailTypeNotSupportedError: If tools is given, or a requested rail type is tool_call or tool_result, which only the IORails engine runs.
  • RailTypeNotConfiguredError: If a requested rail type has no configured flows.

Examples:

Check user input (auto-detected)::
result = await rails.check_async([{"role": "user", "content": "Hello!"}])
if result.status == RailStatus.BLOCKED:
print(f"Blocked by: {result.rail}")
Check bot output with context (auto-detected)::
result = await rails.check_async([
{"role": "user", "content": "Hello!"},
{"role": "assistant", "content": "Hi there!"}
])
Run only input rails explicitly::
result = await rails.check_async(messages, rail_types=[RailType.INPUT])
nemoguardrails.LLMRails.explain() -> nemoguardrails.logging.explain.ExplainInfo

Helper function to return the latest ExplainInfo object.

nemoguardrails.LLMRails.generate(
prompt: typing.Optional[str] = None,
messages: typing.Optional[typing.List[dict]] = None,
options: typing.Optional[typing.Union[dict, nemoguardrails.rails.llm.options.GenerationOptions]] = None,
state: typing.Optional[dict] = None
)

Synchronous version of generate_async.

nemoguardrails.LLMRails.generate_async(
prompt: typing.Optional[str] = None,
messages: typing.Optional[typing.List[dict]] = None,
options: typing.Optional[typing.Union[dict, nemoguardrails.rails.llm.options.GenerationOptions]] = None,
state: typing.Optional[typing.Union[dict, nemoguardrails.colang.v2_x.runtime.flows.State]] = None,
streaming_handler: typing.Optional[nemoguardrails.streaming.StreamingHandler] = None
) -> typing.Union[str, dict, nemoguardrails.rails.llm.options.GenerationResponse, typing.Tuple[dict, dict]]
async

Generate a completion or a next message.

The format for messages is the following:

[
{"role": "context", "content": {"user_name": "John"}},
{"role": "user", "content": "Hello! How are you?"},
{"role": "assistant", "content": "I am fine, thank you!"},
{"role": "event", "event": {"type": "UserSilent"}},
...
]

System messages are not yet supported.

Parameters:

prompt
Optional[str]Defaults to None

The prompt to be used for completion.

messages
Optional[List[dict]]Defaults to None

The history of messages to be used to generate the next message.

options
Optional[Union[dict, GenerationOptions]]Defaults to None

Optional[Union[dict, GenerationOptions ]] — Options specific for the generation.

state
Optional[Union[dict, State]]Defaults to None

The state object that should be used as the starting point.

streaming_handler
Optional[StreamingHandler]Defaults to None

Optional[ StreamingHandler ] — If specified, and the config supports streaming, the provided handler will be used for streaming.

Returns: Union[str, dict, GenerationResponse , Tuple[dict, dict]]

The completion (when a prompt is provided) or the next message.

nemoguardrails.LLMRails.generate_events(
events: typing.List[dict]
) -> typing.List[dict]

Synchronous version of LLMRails.generate_events_async.

nemoguardrails.LLMRails.generate_events_async(
events: typing.List[dict]
) -> typing.List[dict]
async

Generate the next events based on the provided history.

The format for events is the following:

[
{"type": "...", ...},
...
]

Parameters:

events
List[dict]

The history of events to be used to generate the next events.

options

The options to be used for the generation.

Returns: List[dict]

The newly generate event(s).

nemoguardrails.LLMRails.process_events(
events: typing.List[dict],
state: typing.Union[typing.Optional[dict], nemoguardrails.colang.v2_x.runtime.flows.State] = None,
blocking: bool = False
) -> typing.Tuple[typing.List[dict], typing.Union[dict, nemoguardrails.colang.v2_x.runtime.flows.State]]

Synchronous version of LLMRails.process_events_async.

nemoguardrails.LLMRails.process_events_async(
events: typing.List[dict],
state: typing.Union[typing.Optional[dict], nemoguardrails.colang.v2_x.runtime.flows.State] = None,
blocking: bool = False
) -> typing.Tuple[typing.List[dict], typing.Union[dict, nemoguardrails.colang.v2_x.runtime.flows.State]]
async

Process a sequence of events in a given state.

The events will be processed one by one, in the input order.

Parameters:

events
List[dict]

A sequence of events that needs to be processed.

state
Union[Optional[dict], State]Defaults to None

The state that should be used as the starting point. If not provided, a clean state will be used.

Returns: Tuple[List[dict], Union[dict, State]]

(output_events, output_state) Returns a sequence of output events and an output state.

nemoguardrails.LLMRails.register_action(
action: typing.Callable,
name: typing.Optional[str] = None
) -> typing_extensions.Self

Register a custom action for the rails configuration.

nemoguardrails.LLMRails.register_action_param(
name: str,
value: typing.Any
) -> typing_extensions.Self

Registers a custom action parameter.

nemoguardrails.LLMRails.register_embedding_provider(
name: typing.Optional[str] = None
) -> typing_extensions.Self

Register a custom embedding provider.

Parameters:

model
Type[EmbeddingModel]

Type[ EmbeddingModel ] — The embedding model class.

name
strDefaults to None

The name of the embedding engine. If available in the model, it will be used.

Raises:

  • ValueError: If the engine name is not provided and the model does not have an engine name.
  • ValueError: If the model does not have ‘encode’ or ‘encode_async’ methods.
nemoguardrails.LLMRails.register_embedding_search_provider(
name: str
) -> typing_extensions.Self

Register a new embedding search provider.

Parameters:

name
str

The name of the embedding search provider that will be used.

cls
Type[EmbeddingsIndex]

Type[ EmbeddingsIndex ] — The class that will be used to generate and search embedding

nemoguardrails.LLMRails.register_filter(
filter_fn: typing.Callable,
name: typing.Optional[str] = None
) -> typing_extensions.Self

Register a custom filter for the rails configuration.

nemoguardrails.LLMRails.register_output_parser(
output_parser: typing.Callable,
name: str
) -> typing_extensions.Self

Register a custom output parser for the rails configuration.

nemoguardrails.LLMRails.register_prompt_context(
name: str,
value_or_fn: typing.Any
) -> typing_extensions.Self

Register a value to be included in the prompt context.

:name: The name of the variable or function that will be used. :value_or_fn: The value or function that will be used to generate the value.

nemoguardrails.LLMRails.stream_async(
prompt: typing.Optional[str] = None,
messages: typing.Optional[typing.List[dict]] = None,
options: typing.Optional[typing.Union[dict, nemoguardrails.rails.llm.options.GenerationOptions]] = None,
state: typing.Optional[typing.Union[dict, nemoguardrails.colang.v2_x.runtime.flows.State]] = None,
include_metadata: typing.Optional[bool] = False,
generator: typing.Optional[typing.AsyncIterator[typing.Union[str, nemoguardrails.types.ToolCallChunk, nemoguardrails.types.ToolCallDeltaChunk]]] = None,
include_generation_metadata: typing.Optional[bool] = None
) -> typing.AsyncIterator[typing.Union[str, dict, nemoguardrails.types.ToolCallChunk, nemoguardrails.types.ToolCallDeltaChunk]]

Simplified interface for getting directly the streamed tokens from the LLM.

nemoguardrails.LLMRails.update_llm(
)

Replace the main LLM with the provided one.

Parameters:

llm
LLMModel

LLMModel — The new LLM that should be used.

class nemoguardrails.LLMRailsRemovalWarning()

Bases: DeprecationWarning

A feature that is removed together with the LLMRails engine.

class nemoguardrails.LLMResponse(
content: str,
reasoning: typing.Optional[str] = None,
tool_calls: typing.Optional[typing.List[nemoguardrails.types.ToolCall]] = None,
model: typing.Optional[str] = None,
finish_reason: typing.Optional[nemoguardrails.types.FinishReason] = None,
stop_sequence: typing.Optional[str] = None,
request_id: typing.Optional[str] = None,
usage: typing.Optional[nemoguardrails.types.UsageInfo] = None,
provider_metadata: typing.Optional[typing.Dict[str, typing.Any]] = None
)
Dataclass
content
str
finish_reason
Optional[FinishReason] = None
model
Optional[str] = None
provider_metadata
Optional[Dict[str, Any]] = None
reasoning
Optional[str] = None
request_id
Optional[str] = None
stop_sequence
Optional[str] = None
tool_calls
Optional[List[ToolCall]] = None
usage
Optional[UsageInfo] = None
class nemoguardrails.LLMResponseChunk(
delta_content: typing.Optional[str] = None,
delta_reasoning: typing.Optional[str] = None,
delta_tool_calls: typing.Optional[typing.List[nemoguardrails.types.ToolCall]] = None,
model: typing.Optional[str] = None,
finish_reason: typing.Optional[nemoguardrails.types.FinishReason] = None,
request_id: typing.Optional[str] = None,
usage: typing.Optional[nemoguardrails.types.UsageInfo] = None,
provider_metadata: typing.Optional[typing.Dict[str, typing.Any]] = None,
tool_call_deltas: typing.Optional[typing.List[nemoguardrails.types.ToolCallDelta]] = None
)
Dataclass
delta_content
Optional[str] = None
delta_reasoning
Optional[str] = None
delta_tool_calls
Optional[List[ToolCall]] = None
finish_reason
Optional[FinishReason] = None
model
Optional[str] = None
provider_metadata
Optional[Dict[str, Any]] = None
request_id
Optional[str] = None
tool_call_deltas
Optional[List[ToolCallDelta]] = None
usage
Optional[UsageInfo] = None
class nemoguardrails.RailsConfig()

Bases: BaseModel

Configuration object for the models and the rails.

TODO: add typed config for user_messages, bot_messages, and flows.

actions_server_url
Optional[str]
bot_messages
Dict[str, List[str]]
colang_version
str
config_path
Optional[str]
core
CoreConfig
custom_data
Dict
docs
Optional[List[Document]]
enable_multi_step_generation
Optional[bool]
enable_rails_exceptions
bool
event_source_uid
str
flows
List[Union[Dict, Any]]
hooks_server
HooksServerConfig
import_paths
Optional[List[str]]
imported_paths
Optional[Dict[str, str]]
instructions
Optional[List[Instruction]]
knowledge_base
KnowledgeBaseConfig
lowest_temperature
Optional[float]
metrics
MetricsConfig
models
List[Model]
passthrough
Optional[bool]
prompting_mode
Optional[str]
prompts
Optional[List[TaskPrompt]]
rails
Rails
raw_llm_call_action
Optional[str]
sample_conversation
Optional[str]
streaming
bool
tracing
TracingConfig
user_messages
Dict[str, List[str]]
nemoguardrails.RailsConfig.__add__(
other
)

Adds two RailsConfig objects.

nemoguardrails.RailsConfig.check_jailbreak_detection_config(
values
)
classmethod

Validate jailbreak detection configuration against enabled flows.

nemoguardrails.RailsConfig.check_model_exists_for_input_rails(
values
)
classmethod

Make sure we have a model for each input rail where one is provided using $model=<model_type>

nemoguardrails.RailsConfig.check_model_exists_for_output_rails(
values
)
classmethod

Make sure we have a model for each output rail where one is provided using $model=<model_type>

nemoguardrails.RailsConfig.check_model_exists_for_tool_rails() -> nemoguardrails.rails.llm.config.RailsConfig

Make sure we have every model a tool rail’s manifest binds, such as the judge named by $model=

nemoguardrails.RailsConfig.check_output_parser_exists(
values
)
classmethod
nemoguardrails.RailsConfig.check_prompt_exist_for_self_check_rails(
values
)
classmethod
nemoguardrails.RailsConfig.check_streaming_can_apply_output_rewrites(
values
)
classmethod

Reject configs whose output rails transform response with stream_first == True

nemoguardrails.RailsConfig.fill_in_default_values_for_v2_x(
values
)
classmethod
nemoguardrails.RailsConfig.from_content(
colang_content: typing.Optional[str] = None,
yaml_content: typing.Optional[str] = None,
config: typing.Optional[dict] = None
)
classmethod

Loads a configuration from the provided colang/YAML content/config dict.

nemoguardrails.RailsConfig.from_path(
config_path: str
)
classmethod

Loads a configuration from a given path.

Supports loading a from a single file, or from a directory.

nemoguardrails.RailsConfig.parse_object(
obj
)
classmethod

Parses a configuration object from a given dictionary.

nemoguardrails.RailsConfig.validate_models_api_key_env_var(
models
)
classmethod

Model API Key Env var must be set to make LLM calls

nemoguardrails.RailsConfig.warn_unknown_keys(
values: typing.Any,
handler: pydantic.ModelWrapValidatorHandler[nemoguardrails.rails.llm.config.RailsConfig]
classmethod
class nemoguardrails.Role

Bases: enum.Enum

ASSISTANT
= 'assistant'
SYSTEM
= 'system'
TOOL
= 'tool'
USER
= 'user'
class nemoguardrails.ToolCall(
id: str,
type: str = 'function',
function: nemoguardrails.types.ToolCallFunction = (lambda: ToolCallFunction(n...
)
Dataclass
function
ToolCallFunction
id
str
type
str = 'function'
nemoguardrails.ToolCall.to_dict() -> typing.Dict[str, typing.Any]
class nemoguardrails.ToolCallChunk(
tool_calls: typing.List[nemoguardrails.types.ToolCall]
)
Dataclass

A structured stream event containing completed tool calls, not argument fragments.

tool_calls
List[ToolCall]
class nemoguardrails.ToolCallDelta(
index: int,
id: typing.Optional[str] = None,
type: typing.Optional[str] = None,
function: typing.Optional[nemoguardrails.types.ToolCallFunctionDelta] = None
)
Dataclass
function
Optional[ToolCallFunctionDelta] = None
id
Optional[str] = None
index
int
type
Optional[str] = None
nemoguardrails.ToolCallDelta.to_dict() -> typing.Dict[str, typing.Any]
class nemoguardrails.ToolCallDeltaChunk(
tool_calls: typing.List[nemoguardrails.types.ToolCallDelta] = list(),
finish_reason: typing.Optional[nemoguardrails.types.FinishReason] = None
)
Dataclass
finish_reason
Optional[FinishReason] = None
tool_calls
List[ToolCallDelta] = field(default_factory=list)
class nemoguardrails.ToolCallFunction(
name: str,
arguments: typing.Dict[str, typing.Any]
)
Dataclass
arguments
Dict[str, Any]
name
str
class nemoguardrails.ToolCallFunctionDelta(
name: typing.Optional[str] = None,
arguments: typing.Optional[str] = None
)
Dataclass
arguments
Optional[str] = None
name
Optional[str] = None
class nemoguardrails.UsageInfo(
input_tokens: int = 0,
output_tokens: int = 0,
total_tokens: int = 0,
reasoning_tokens: typing.Optional[int] = None,
cached_tokens: typing.Optional[int] = None
)
Dataclass
cached_tokens
Optional[int] = None
input_tokens
int = 0
output_tokens
int = 0
reasoning_tokens
Optional[int] = None
total_tokens
int = 0
nemoguardrails.get_default_framework() -> str
nemoguardrails.register_framework(
name: str,
) -> None

Register an LLMFramework instance under name.

Validates two invariants before storing the instance:

  1. The object structurally matches the LLMFramework Protocol (create_model, register_provider, get_provider_names, reset).
  2. Its reset attribute is an async coroutine function. The registry awaits it directly during shutdown / test teardown.

Raises:

  • ValueError: a framework is already registered under name.
  • TypeError: framework does not implement the LLMFramework Protocol, or its reset is not an async coroutine function.
nemoguardrails.register_provider(
name: str,
provider_cls: typing.Any
) -> None
nemoguardrails.set_default_framework(
name: str
) -> None
nemoguardrails.types.FinishReason = Literal['stop', 'length', 'tool_calls', 'content_filter', 'error', 'other']
nemoguardrails.__version__ = version('nemoguardrails')
nemoguardrails._use_guardrails_wrapper = os.environ.get('NEMO_GUARDRAILS_IORAILS_ENGINE', '').lower() in ('true', '1', 'y...