nemoguardrails.testing
Public testing surface for NeMo Guardrails.
This subpackage ships utilities that help users write fast, deterministic tests for their guardrails configurations. The two main building blocks are:
FakeLLMModel: a scriptable implementation of theLLMModelprotocol that returns canned responses.RecordingHTTPClient: a scriptable HTTP client that records requests and returns queued responses.TestChat: an ergonomic helper for asserting bot replies against a scripted conversation.
Pytest fixtures are exposed via the nemoguardrails.testing.fixtures plugin.
Add it to your conftest.py to opt in:
Submodules
nemoguardrails.testing.chat_harnessnemoguardrails.testing.fake_modelnemoguardrails.testing.fixturesnemoguardrails.testing.http
Package Contents
Classes
API
Framework-agnostic fake LLM for testing. Implements the LLMModel protocol.
Parameters:
A list of plain string responses. Each call to
generate_async (or stream_async) consumes the next
entry. Mutually exclusive with llm_responses.
Optional[List[ LLMResponse ]] — A list of LLMResponse
objects. Useful when tool calls or structured fields need to be
asserted. Takes precedence over responses when provided.
An exception instance to raise on every generation, useful for exercising error-handling paths.
Optional list of token usage dictionaries (one per
response). Each entry may include prompt_tokens,
completion_tokens and total_tokens keys.
When True, populate
LLMResponse.usage from token_usage.
Record requests and return queued responses or exceptions in order.
Record a close call without discarding captured requests.
Record a request and consume the next queued result.
Raises:
RuntimeError: If no queued result is available.BaseException: The queued exception, when the next result is an exception instance.
Helper class for easily writing tests.
Usage: config = RailsConfig.from_path(…) chat = TestChat( config, llm_completions=[ “Hello! How can I help you today?”, ], )
chat.user(“Hello! How are you?”) chat.bot(“Hello! How can I help you today?“)