> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/guardrails/llms.txt.
> For full documentation content, see https://docs.nvidia.com/nemo/guardrails/llms-full.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/guardrails/_mcp/server.

# nemoguardrails.types

Public LLM types for NeMo Guardrails.

This module defines the stable, public dataclasses and Protocols that make
up the LLM interop surface for NeMo Guardrails. The types are implemented
as plain Python dataclasses (not Pydantic) so they remain lightweight and
introduce no additional runtime dependencies for downstream integrators.

The public surface defined here is stable across minor versions: breaking
changes are reserved for major version bumps. Custom `LLMModel` and
`LLMFramework` implementations should import the relevant types from
this module so they stay aligned with the canonical definitions.

## Module Contents

### Classes

| Name                                                         | Description                                    |
| ------------------------------------------------------------ | ---------------------------------------------- |
| [`ChatMessage`](#nemoguardrails-types-ChatMessage)           | -                                              |
| [`LLMFramework`](#nemoguardrails-types-LLMFramework)         | Protocol for pluggable LLM framework backends. |
| [`LLMModel`](#nemoguardrails-types-LLMModel)                 | Protocol that all LLM backends must implement. |
| [`LLMResponse`](#nemoguardrails-types-LLMResponse)           | -                                              |
| [`LLMResponseChunk`](#nemoguardrails-types-LLMResponseChunk) | -                                              |
| [`Role`](#nemoguardrails-types-Role)                         | -                                              |
| [`ToolCall`](#nemoguardrails-types-ToolCall)                 | -                                              |
| [`ToolCallFunction`](#nemoguardrails-types-ToolCallFunction) | -                                              |
| [`UsageInfo`](#nemoguardrails-types-UsageInfo)               | -                                              |

### Data

[`FinishReason`](#nemoguardrails-types-FinishReason)

[`_ROLE_ALIASES`](#nemoguardrails-types-_ROLE_ALIASES)

[`_STANDARD_MESSAGE_KEYS`](#nemoguardrails-types-_STANDARD_MESSAGE_KEYS)

[`__all__`](#nemoguardrails-types-__all__)

### API

```python
class nemoguardrails.types.ChatMessage(
    role: nemoguardrails.types.Role,
    content: typing.Optional[typing.Union[str, typing.List[typing.Dict[str, typing.Any]]]] = None,
    tool_calls: typing.Optional[typing.List[nemoguardrails.types.ToolCall]] = None,
    tool_call_id: typing.Optional[str] = None,
    name: typing.Optional[str] = None,
    provider_metadata: typing.Dict[str, typing.Any] = dict()
)
```

Dataclass

```python
nemoguardrails.types.ChatMessage.from_assistant(
    content: str,
    kwargs = {}
) -> nemoguardrails.types.ChatMessage
```

classmethod

```python
nemoguardrails.types.ChatMessage.from_dict(
    d: typing.Dict[str, typing.Any]
) -> nemoguardrails.types.ChatMessage
```

classmethod

Create a ChatMessage from a dict.

Accepts both the canonical nested tool call format
(`{"function": {"name": ..., "arguments": ...}}`) and the legacy
flat format (`{"name": ..., "args": ...}`). JSON string arguments
are parsed automatically. Role aliases like "bot", "human", and
"developer" are mapped to canonical Role values. Unknown keys are
captured into `provider_metadata`.

```python
nemoguardrails.types.ChatMessage.from_system(
    content: str,
    kwargs = {}
) -> nemoguardrails.types.ChatMessage
```

classmethod

```python
nemoguardrails.types.ChatMessage.from_tool(
    content: str,
    tool_call_id: str,
    kwargs = {}
) -> nemoguardrails.types.ChatMessage
```

classmethod

```python
nemoguardrails.types.ChatMessage.from_user(
    content: str,
    kwargs = {}
) -> nemoguardrails.types.ChatMessage
```

classmethod

```python
nemoguardrails.types.ChatMessage.to_dict() -> typing.Dict[str, typing.Any]
```

```python
class nemoguardrails.types.LLMFramework()
```

Protocol

Protocol for pluggable LLM framework backends.

Each framework (LangChain, LiteLLM, etc.) implements this protocol to
provide a factory for creating `LLMModel` instances and managing
its own set of providers.

`model_kwargs` carries all provider-specific configuration. Framework
implementations extract what they need (e.g. LangChain pops `mode`
to choose between chat and text completion models).

```python
nemoguardrails.types.LLMFramework.create_model(
    model_name: str,
    provider_name: str,
    model_kwargs: typing.Optional[typing.Dict[str, typing.Any]] = None
) -> nemoguardrails.types.LLMModel
```

```python
nemoguardrails.types.LLMFramework.get_provider_names() -> typing.List[str]
```

```python
nemoguardrails.types.LLMFramework.register_provider(
    name: str,
    provider_cls: typing.Any
) -> None
```

```python
nemoguardrails.types.LLMFramework.reset() -> None
```

async

Release all framework-owned resources and clear all registered state.

Implementations should close any pooled connections, clear registered
providers, and return the framework to its initial state. Callers can
continue using the instance after reset; new resources will be created
on demand.

```python
class nemoguardrails.types.LLMModel()
```

Protocol

Protocol that all LLM backends must implement.

Adapters wrap provider-specific SDKs (LangChain, LiteLLM, OpenAI, etc.)
behind this interface so the core pipeline remains framework-agnostic.

`prompt` accepts either a plain string or a list of `ChatMessage`
objects. Adapters convert `ChatMessage` to whatever their SDK expects.
`**kwargs` are forwarded to the underlying SDK (e.g. temperature,
max\_tokens).

```python
nemoguardrails.types.LLMModel.generate_async(
    prompt: typing.Union[str, typing.List[nemoguardrails.types.ChatMessage]],
    stop: typing.Optional[typing.List[str]] = None,
    kwargs = {}
) -> nemoguardrails.types.LLMResponse
```

async

```python
nemoguardrails.types.LLMModel.stream_async(
    prompt: typing.Union[str, typing.List[nemoguardrails.types.ChatMessage]],
    stop: typing.Optional[typing.List[str]] = None,
    kwargs = {}
) -> typing.AsyncIterator[nemoguardrails.types.LLMResponseChunk]
```

Implementations must be async generator functions (use `yield`).

```python
class nemoguardrails.types.LLMResponse(
    content: str,
    reasoning: typing.Optional[str] = None,
    tool_calls: typing.Optional[typing.List[nemoguardrails.types.ToolCall]] = None,
    model: typing.Optional[str] = None,
    finish_reason: typing.Optional[nemoguardrails.types.FinishReason] = None,
    stop_sequence: typing.Optional[str] = None,
    request_id: typing.Optional[str] = None,
    usage: typing.Optional[nemoguardrails.types.UsageInfo] = None,
    provider_metadata: typing.Optional[typing.Dict[str, typing.Any]] = None
)
```

Dataclass

```python
class nemoguardrails.types.LLMResponseChunk(
    delta_content: typing.Optional[str] = None,
    delta_reasoning: typing.Optional[str] = None,
    delta_tool_calls: typing.Optional[typing.List[nemoguardrails.types.ToolCall]] = None,
    model: typing.Optional[str] = None,
    finish_reason: typing.Optional[nemoguardrails.types.FinishReason] = None,
    request_id: typing.Optional[str] = None,
    usage: typing.Optional[nemoguardrails.types.UsageInfo] = None,
    provider_metadata: typing.Optional[typing.Dict[str, typing.Any]] = None
)
```

Dataclass

```python
class nemoguardrails.types.Role
```

**Bases:** `enum.Enum`

```python
class nemoguardrails.types.ToolCall(
    id: str,
    type: str = 'function',
    function: nemoguardrails.types.ToolCallFunction = (lambda: ToolCallFunction(n...
)
```

Dataclass

```python
nemoguardrails.types.ToolCall.to_dict() -> typing.Dict[str, typing.Any]
```

```python
class nemoguardrails.types.ToolCallFunction(
    name: str,
    arguments: typing.Dict[str, typing.Any]
)
```

Dataclass

```python
class nemoguardrails.types.UsageInfo(
    input_tokens: int = 0,
    output_tokens: int = 0,
    total_tokens: int = 0,
    reasoning_tokens: typing.Optional[int] = None,
    cached_tokens: typing.Optional[int] = None
)
```

Dataclass

```python
nemoguardrails.types.FinishReason = Literal['stop', 'length', 'tool_calls', 'content_filter', 'error', 'other']
```

```python
nemoguardrails.types._ROLE_ALIASES = {'bot': Role.ASSISTANT, 'assistant': Role.ASSISTANT, 'human': Role.USER, 'user':...
```

```python
nemoguardrails.types._STANDARD_MESSAGE_KEYS = {'role', 'content', 'tool_calls', 'tool_call_id', 'name', 'provider_metadata'}
```

```python
nemoguardrails.types.__all__ = ['ChatMessage', 'FinishReason', 'LLMFramework', 'LLMModel', 'LLMResponse', 'LLMR...
```