> This page is for version v0.25.0.
> For other versions, use one of these documentation indexes:
> - Latest (default): https://docs.nvidia.com/nemo/guardrails/latest/llms.txt
> - v0.25.0: https://docs.nvidia.com/nemo/guardrails/v0.25.0/llms.txt
> - v0.24.1: https://docs.nvidia.com/nemo/guardrails/v0.24.1/llms.txt
> - v0.24.0: https://docs.nvidia.com/nemo/guardrails/v0.24.0/llms.txt
> - v0.23.0: https://docs.nvidia.com/nemo/guardrails/v0.23.0/llms.txt
> - v0.22.0: https://docs.nvidia.com/nemo/guardrails/v0.22.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/guardrails/llms.txt.

# nemoguardrails.llm

## Subpackages

* **[`nemoguardrails.llm.cache`](/nemo/guardrails/v0.25.0/guardrails-python-sdk/nemoguardrails/llm/cache)**
* **[`nemoguardrails.llm.clients`](/nemo/guardrails/v0.25.0/guardrails-python-sdk/nemoguardrails/llm/clients)**
* **[`nemoguardrails.llm.frameworks`](/nemo/guardrails/v0.25.0/guardrails-python-sdk/nemoguardrails/llm/frameworks)**

## Submodules

* **[`nemoguardrails.llm.call`](/nemo/guardrails/v0.25.0/guardrails-python-sdk/nemoguardrails/llm/call)**
* **[`nemoguardrails.llm.completion_parsing`](/nemo/guardrails/v0.25.0/guardrails-python-sdk/nemoguardrails/llm/completion_parsing)**
* **[`nemoguardrails.llm.constants`](/nemo/guardrails/v0.25.0/guardrails-python-sdk/nemoguardrails/llm/constants)**
* **[`nemoguardrails.llm.filters`](/nemo/guardrails/v0.25.0/guardrails-python-sdk/nemoguardrails/llm/filters)**
* **[`nemoguardrails.llm.openai_reasoning`](/nemo/guardrails/v0.25.0/guardrails-python-sdk/nemoguardrails/llm/openai_reasoning)**
* **[`nemoguardrails.llm.output_parsers`](/nemo/guardrails/v0.25.0/guardrails-python-sdk/nemoguardrails/llm/output_parsers)**
* **[`nemoguardrails.llm.prompts`](/nemo/guardrails/v0.25.0/guardrails-python-sdk/nemoguardrails/llm/prompts)**
* **[`nemoguardrails.llm.providers`](/nemo/guardrails/v0.25.0/guardrails-python-sdk/nemoguardrails/llm/providers)**
* **[`nemoguardrails.llm.taskmanager`](/nemo/guardrails/v0.25.0/guardrails-python-sdk/nemoguardrails/llm/taskmanager)**
* **[`nemoguardrails.llm.telemetry`](/nemo/guardrails/v0.25.0/guardrails-python-sdk/nemoguardrails/llm/telemetry)**
* **[`nemoguardrails.llm.types`](/nemo/guardrails/v0.25.0/guardrails-python-sdk/nemoguardrails/llm/types)**

## Package Contents

### Classes

| Name                                                         | Description                                    |
| ------------------------------------------------------------ | ---------------------------------------------- |
| [`ChatMessage`](#nemoguardrails-types-ChatMessage)           | -                                              |
| [`LLMFramework`](#nemoguardrails-types-LLMFramework)         | Protocol for pluggable LLM framework backends. |
| [`LLMModel`](#nemoguardrails-types-LLMModel)                 | Protocol that all LLM backends must implement. |
| [`LLMResponse`](#nemoguardrails-types-LLMResponse)           | -                                              |
| [`LLMResponseChunk`](#nemoguardrails-types-LLMResponseChunk) | -                                              |
| [`Role`](#nemoguardrails-types-Role)                         | -                                              |
| [`ToolCall`](#nemoguardrails-types-ToolCall)                 | -                                              |
| [`ToolCallFunction`](#nemoguardrails-types-ToolCallFunction) | -                                              |
| [`UsageInfo`](#nemoguardrails-types-UsageInfo)               | -                                              |

### Functions

| Name                                                                                     | Description                                       |
| ---------------------------------------------------------------------------------------- | ------------------------------------------------- |
| [`get_default_framework`](#nemoguardrails-llm-frameworks-registry-get_default_framework) | -                                                 |
| [`register_framework`](#nemoguardrails-llm-frameworks-registry-register_framework)       | Register an `LLMFramework` instance under `name`. |
| [`register_provider`](#nemoguardrails-llm-providers-register_provider)                   | -                                                 |
| [`set_default_framework`](#nemoguardrails-llm-frameworks-registry-set_default_framework) | -                                                 |

### Data

[`FinishReason`](#nemoguardrails-types-FinishReason)

### API

```python
class nemoguardrails.ChatMessage(
    role: nemoguardrails.types.Role,
    content: typing.Optional[typing.Union[str, typing.List[typing.Dict[str, typing.Any]]]] = None,
    tool_calls: typing.Optional[typing.List[nemoguardrails.types.ToolCall]] = None,
    tool_call_id: typing.Optional[str] = None,
    name: typing.Optional[str] = None,
    provider_metadata: typing.Dict[str, typing.Any] = dict()
)
```

Dataclass

**`content`** `Optional[Union[str, List[Dict[str, Any]]]] = None`

---

**`name`** `Optional[str] = None`

---

**`provider_metadata`** `Dict[str, Any] = field(default_factory=dict)`

---

**`role`** `Role`

---

**`tool_call_id`** `Optional[str] = None`

---

**`tool_calls`** `Optional[List[ToolCall]] = None`

---

```python
nemoguardrails.ChatMessage.from_assistant(
    content: str,
    kwargs = {}
) -> nemoguardrails.types.ChatMessage
```

classmethod

```python
nemoguardrails.ChatMessage.from_dict(
    d: typing.Dict[str, typing.Any]
) -> nemoguardrails.types.ChatMessage
```

classmethod

Create a ChatMessage from a dict.

Accepts both the canonical nested tool call format
(`{"function": {"name": ..., "arguments": ...}}`) and the legacy
flat format (`{"name": ..., "args": ...}`). JSON string arguments
are parsed automatically. Role aliases like "bot", "human", and
"developer" are mapped to canonical Role values. Unknown keys are
captured into `provider_metadata`.

```python
nemoguardrails.ChatMessage.from_system(
    content: str,
    kwargs = {}
) -> nemoguardrails.types.ChatMessage
```

classmethod

```python
nemoguardrails.ChatMessage.from_tool(
    content: str,
    tool_call_id: str,
    kwargs = {}
) -> nemoguardrails.types.ChatMessage
```

classmethod

```python
nemoguardrails.ChatMessage.from_user(
    content: str,
    kwargs = {}
) -> nemoguardrails.types.ChatMessage
```

classmethod

```python
nemoguardrails.ChatMessage.to_dict() -> typing.Dict[str, typing.Any]
```

```python
class nemoguardrails.LLMFramework()
```

Protocol

Protocol for pluggable LLM framework backends.

Each framework (LangChain, LiteLLM, etc.) implements this protocol to
provide a factory for creating `LLMModel` instances and managing
its own set of providers.

`model_kwargs` carries all provider-specific configuration. Framework
implementations extract what they need (e.g. LangChain pops `mode`
to choose between chat and text completion models).

```python
nemoguardrails.LLMFramework.create_model(
    model_name: str,
    provider_name: str,
    model_kwargs: typing.Optional[typing.Dict[str, typing.Any]] = None
) -> nemoguardrails.types.LLMModel
```

```python
nemoguardrails.LLMFramework.get_provider_names() -> typing.List[str]
```

```python
nemoguardrails.LLMFramework.register_provider(
    name: str,
    provider_cls: typing.Any
) -> None
```

```python
nemoguardrails.LLMFramework.reset() -> None
```

async

Release all framework-owned resources and clear all registered state.

Implementations should close any pooled connections, clear registered
providers, and return the framework to its initial state. Callers can
continue using the instance after reset; new resources will be created
on demand.

```python
class nemoguardrails.LLMModel()
```

Protocol

Protocol that all LLM backends must implement.

Adapters wrap provider-specific SDKs (LangChain, LiteLLM, OpenAI, etc.)
behind this interface so the core pipeline remains framework-agnostic.

`prompt` accepts either a plain string or a list of `ChatMessage`
objects. Adapters convert `ChatMessage` to whatever their SDK expects.
`**kwargs` are forwarded to the underlying SDK (e.g. temperature,
max_tokens).

**`model_name`** `str`

---

**`provider_name`** `Optional[str]`

---

**`provider_url`** `Optional[str]`

---

```python
nemoguardrails.LLMModel.generate_async(
    prompt: typing.Union[str, typing.List[nemoguardrails.types.ChatMessage]],
    stop: typing.Optional[typing.List[str]] = None,
    kwargs = {}
) -> nemoguardrails.types.LLMResponse
```

async

```python
nemoguardrails.LLMModel.stream_async(
    prompt: typing.Union[str, typing.List[nemoguardrails.types.ChatMessage]],
    stop: typing.Optional[typing.List[str]] = None,
    kwargs = {}
) -> typing.AsyncIterator[nemoguardrails.types.LLMResponseChunk]
```

Implementations must be async generator functions (use `yield`).

```python
class nemoguardrails.LLMResponse(
    content: str,
    reasoning: typing.Optional[str] = None,
    tool_calls: typing.Optional[typing.List[nemoguardrails.types.ToolCall]] = None,
    model: typing.Optional[str] = None,
    finish_reason: typing.Optional[nemoguardrails.types.FinishReason] = None,
    stop_sequence: typing.Optional[str] = None,
    request_id: typing.Optional[str] = None,
    usage: typing.Optional[nemoguardrails.types.UsageInfo] = None,
    provider_metadata: typing.Optional[typing.Dict[str, typing.Any]] = None
)
```

Dataclass

**`content`** `str`

---

**`finish_reason`** `Optional[FinishReason] = None`

---

**`model`** `Optional[str] = None`

---

**`provider_metadata`** `Optional[Dict[str, Any]] = None`

---

**`reasoning`** `Optional[str] = None`

---

**`request_id`** `Optional[str] = None`

---

**`stop_sequence`** `Optional[str] = None`

---

**`tool_calls`** `Optional[List[ToolCall]] = None`

---

**`usage`** `Optional[UsageInfo] = None`

---

```python
class nemoguardrails.LLMResponseChunk(
    delta_content: typing.Optional[str] = None,
    delta_reasoning: typing.Optional[str] = None,
    delta_tool_calls: typing.Optional[typing.List[nemoguardrails.types.ToolCall]] = None,
    model: typing.Optional[str] = None,
    finish_reason: typing.Optional[nemoguardrails.types.FinishReason] = None,
    request_id: typing.Optional[str] = None,
    usage: typing.Optional[nemoguardrails.types.UsageInfo] = None,
    provider_metadata: typing.Optional[typing.Dict[str, typing.Any]] = None,
    tool_call_deltas: typing.Optional[typing.List[nemoguardrails.types.ToolCallDelta]] = None
)
```

Dataclass

**`delta_content`** `Optional[str] = None`

---

**`delta_reasoning`** `Optional[str] = None`

---

**`delta_tool_calls`** `Optional[List[ToolCall]] = None`

---

**`finish_reason`** `Optional[FinishReason] = None`

---

**`model`** `Optional[str] = None`

---

**`provider_metadata`** `Optional[Dict[str, Any]] = None`

---

**`request_id`** `Optional[str] = None`

---

**`tool_call_deltas`** `Optional[List[ToolCallDelta]] = None`

---

**`usage`** `Optional[UsageInfo] = None`

---

```python
class nemoguardrails.Role
```

**Bases:** `enum.Enum`

**`ASSISTANT`** `= 'assistant'`

---

**`SYSTEM`** `= 'system'`

---

**`TOOL`** `= 'tool'`

---

**`USER`** `= 'user'`

---

```python
class nemoguardrails.ToolCall(
    id: str,
    type: str = 'function',
    function: nemoguardrails.types.ToolCallFunction = (lambda: ToolCallFunction(n...
)
```

Dataclass

**`function`** `ToolCallFunction`

---

**`id`** `str`

---

**`type`** `str = 'function'`

---

```python
nemoguardrails.ToolCall.to_dict() -> typing.Dict[str, typing.Any]
```

```python
class nemoguardrails.ToolCallFunction(
    name: str,
    arguments: typing.Dict[str, typing.Any]
)
```

Dataclass

**`arguments`** `Dict[str, Any]`

---

**`name`** `str`

---

```python
class nemoguardrails.UsageInfo(
    input_tokens: int = 0,
    output_tokens: int = 0,
    total_tokens: int = 0,
    reasoning_tokens: typing.Optional[int] = None,
    cached_tokens: typing.Optional[int] = None
)
```

Dataclass

**`cached_tokens`** `Optional[int] = None`

---

**`input_tokens`** `int = 0`

---

**`output_tokens`** `int = 0`

---

**`reasoning_tokens`** `Optional[int] = None`

---

**`total_tokens`** `int = 0`

---

```python
nemoguardrails.get_default_framework() -> str
```

```python
nemoguardrails.register_framework(
    name: str,
    framework: nemoguardrails.types.LLMFramework
) -> None
```

Register an `LLMFramework` instance under `name`.

Validates two invariants before storing the instance:

1. The object structurally matches the `LLMFramework` Protocol
   (`create_model`, `register_provider`, `get_provider_names`, `reset`).
2. Its `reset` attribute is an `async` coroutine function. The registry
   awaits it directly during shutdown / test teardown.

**Raises:**

* `ValueError`: a framework is already registered under `name`.
* `TypeError`: `framework` does not implement the `LLMFramework`
  Protocol, or its `reset` is not an async coroutine function.

```python
nemoguardrails.register_provider(
    name: str,
    provider_cls: typing.Any
) -> None
```

```python
nemoguardrails.set_default_framework(
    name: str
) -> None
```

```python
nemoguardrails.types.FinishReason = Literal['stop', 'length', 'tool_calls', 'content_filter', 'error', 'other']
```