ReferencePython SDK ReferenceNemoguardrails

nemoguardrails.types

View as Markdown

Public LLM types for NeMo Guardrails.

This module defines the stable, public dataclasses and Protocols that make up the LLM interop surface for NeMo Guardrails. The types are implemented as plain Python dataclasses (not Pydantic) so they remain lightweight and introduce no additional runtime dependencies for downstream integrators.

The public surface defined here is stable across minor versions: breaking changes are reserved for major version bumps. Custom LLMModel and LLMFramework implementations should import the relevant types from this module so they stay aligned with the canonical definitions.

Module Contents

Classes

NameDescription
ChatMessage-
LLMFrameworkProtocol for pluggable LLM framework backends.
LLMModelProtocol that all LLM backends must implement.
LLMResponse-
LLMResponseChunk-
Role-
ToolCall-
ToolCallFunction-
UsageInfo-

Data

FinishReason

_ROLE_ALIASES

_STANDARD_MESSAGE_KEYS

__all__

API

class nemoguardrails.types.ChatMessage(
    role: nemoguardrails.types.Role,
    content: typing.Optional[typing.Union[str, typing.List[typing.Dict[str, typing.Any]]]] = None,
    tool_calls: typing.Optional[typing.List[nemoguardrails.types.ToolCall]] = None,
    tool_call_id: typing.Optional[str] = None,
    name: typing.Optional[str] = None,
    provider_metadata: typing.Dict[str, typing.Any] = dict()
)
Dataclass
content
Optional[Union[str, List[Dict[str, Any]]]] = None
name
Optional[str] = None
provider_metadata
Dict[str, Any] = field(default_factory=dict)
role
Role
tool_call_id
Optional[str] = None
tool_calls
Optional[List[ToolCall]] = None
nemoguardrails.types.ChatMessage.from_assistant(
    content: str,
    kwargs = {}
) -> nemoguardrails.types.ChatMessage
classmethod
nemoguardrails.types.ChatMessage.from_dict(
    d: typing.Dict[str, typing.Any]
) -> nemoguardrails.types.ChatMessage
classmethod

Create a ChatMessage from a dict.

Accepts both the canonical nested tool call format ({"function": {"name": ..., "arguments": ...}}) and the legacy flat format ({"name": ..., "args": ...}). JSON string arguments are parsed automatically. Role aliases like “bot”, “human”, and “developer” are mapped to canonical Role values. Unknown keys are captured into provider_metadata.

nemoguardrails.types.ChatMessage.from_system(
    content: str,
    kwargs = {}
) -> nemoguardrails.types.ChatMessage
classmethod
nemoguardrails.types.ChatMessage.from_tool(
    content: str,
    tool_call_id: str,
    kwargs = {}
) -> nemoguardrails.types.ChatMessage
classmethod
classmethod
nemoguardrails.types.ChatMessage.to_dict() -> typing.Dict[str, typing.Any]
class nemoguardrails.types.LLMFramework()
Protocol

Protocol for pluggable LLM framework backends.

Each framework (LangChain, LiteLLM, etc.) implements this protocol to provide a factory for creating LLMModel instances and managing its own set of providers.

model_kwargs carries all provider-specific configuration. Framework implementations extract what they need (e.g. LangChain pops mode to choose between chat and text completion models).

nemoguardrails.types.LLMFramework.create_model(
    model_name: str,
    provider_name: str,
    model_kwargs: typing.Optional[typing.Dict[str, typing.Any]] = None
) -> nemoguardrails.types.LLMModel
nemoguardrails.types.LLMFramework.get_provider_names() -> typing.List[str]
nemoguardrails.types.LLMFramework.register_provider(
    name: str,
    provider_cls: typing.Any
) -> None
nemoguardrails.types.LLMFramework.reset() -> None
async

Release all framework-owned resources and clear all registered state.

Implementations should close any pooled connections, clear registered providers, and return the framework to its initial state. Callers can continue using the instance after reset; new resources will be created on demand.

class nemoguardrails.types.LLMModel()
Protocol

Protocol that all LLM backends must implement.

Adapters wrap provider-specific SDKs (LangChain, LiteLLM, OpenAI, etc.) behind this interface so the core pipeline remains framework-agnostic.

prompt accepts either a plain string or a list of ChatMessage objects. Adapters convert ChatMessage to whatever their SDK expects. **kwargs are forwarded to the underlying SDK (e.g. temperature, max_tokens).

model_name
str
provider_name
Optional[str]
provider_url
Optional[str]
nemoguardrails.types.LLMModel.generate_async(
    prompt: typing.Union[str, typing.List[nemoguardrails.types.ChatMessage]],
    stop: typing.Optional[typing.List[str]] = None,
    kwargs = {}
) -> nemoguardrails.types.LLMResponse
async
nemoguardrails.types.LLMModel.stream_async(
    prompt: typing.Union[str, typing.List[nemoguardrails.types.ChatMessage]],
    stop: typing.Optional[typing.List[str]] = None,
    kwargs = {}
) -> typing.AsyncIterator[nemoguardrails.types.LLMResponseChunk]

Implementations must be async generator functions (use yield).

class nemoguardrails.types.LLMResponse(
    content: str,
    reasoning: typing.Optional[str] = None,
    tool_calls: typing.Optional[typing.List[nemoguardrails.types.ToolCall]] = None,
    model: typing.Optional[str] = None,
    finish_reason: typing.Optional[nemoguardrails.types.FinishReason] = None,
    stop_sequence: typing.Optional[str] = None,
    request_id: typing.Optional[str] = None,
    usage: typing.Optional[nemoguardrails.types.UsageInfo] = None,
    provider_metadata: typing.Optional[typing.Dict[str, typing.Any]] = None
)
Dataclass
content
str
finish_reason
Optional[FinishReason] = None
model
Optional[str] = None
provider_metadata
Optional[Dict[str, Any]] = None
reasoning
Optional[str] = None
request_id
Optional[str] = None
stop_sequence
Optional[str] = None
tool_calls
Optional[List[ToolCall]] = None
usage
Optional[UsageInfo] = None
class nemoguardrails.types.LLMResponseChunk(
    delta_content: typing.Optional[str] = None,
    delta_reasoning: typing.Optional[str] = None,
    delta_tool_calls: typing.Optional[typing.List[nemoguardrails.types.ToolCall]] = None,
    model: typing.Optional[str] = None,
    finish_reason: typing.Optional[nemoguardrails.types.FinishReason] = None,
    request_id: typing.Optional[str] = None,
    usage: typing.Optional[nemoguardrails.types.UsageInfo] = None,
    provider_metadata: typing.Optional[typing.Dict[str, typing.Any]] = None
)
Dataclass
delta_content
Optional[str] = None
delta_reasoning
Optional[str] = None
delta_tool_calls
Optional[List[ToolCall]] = None
finish_reason
Optional[FinishReason] = None
model
Optional[str] = None
provider_metadata
Optional[Dict[str, Any]] = None
request_id
Optional[str] = None
usage
Optional[UsageInfo] = None
class nemoguardrails.types.Role

Bases: enum.Enum

ASSISTANT
= 'assistant'
SYSTEM
= 'system'
TOOL
= 'tool'
USER
= 'user'
class nemoguardrails.types.ToolCall(
    id: str,
    type: str = 'function',
    function: nemoguardrails.types.ToolCallFunction = (lambda: ToolCallFunction(n...
)
Dataclass
function
ToolCallFunction
id
str
type
str = 'function'
nemoguardrails.types.ToolCall.to_dict() -> typing.Dict[str, typing.Any]
class nemoguardrails.types.ToolCallFunction(
    name: str,
    arguments: typing.Dict[str, typing.Any]
)
Dataclass
arguments
Dict[str, Any]
name
str
class nemoguardrails.types.UsageInfo(
    input_tokens: int = 0,
    output_tokens: int = 0,
    total_tokens: int = 0,
    reasoning_tokens: typing.Optional[int] = None,
    cached_tokens: typing.Optional[int] = None
)
Dataclass
cached_tokens
Optional[int] = None
input_tokens
int = 0
output_tokens
int = 0
reasoning_tokens
Optional[int] = None
total_tokens
int = 0
nemoguardrails.types.FinishReason = Literal['stop', 'length', 'tool_calls', 'content_filter', 'error', 'other']
 
nemoguardrails.types._ROLE_ALIASES = {'bot': Role.ASSISTANT, 'assistant': Role.ASSISTANT, 'human': Role.USER, 'user':...
 
nemoguardrails.types._STANDARD_MESSAGE_KEYS = {'role', 'content', 'tool_calls', 'tool_call_id', 'name', 'provider_metadata'}
 
nemoguardrails.types.__all__ = ['ChatMessage', 'FinishReason', 'LLMFramework', 'LLMModel', 'LLMResponse', 'LLMR...