> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/guardrails/llms.txt.
> For full documentation content, see https://docs.nvidia.com/nemo/guardrails/llms-full.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/guardrails/_mcp/server.

# nemoguardrails.eval.models

## Module Contents

### Classes

| Name                                                                         | Description                                                           |
| ---------------------------------------------------------------------------- | --------------------------------------------------------------------- |
| [`ComplianceCheckLog`](#nemoguardrails-eval-models-ComplianceCheckLog)       | Detailed log about a compliance check.                                |
| [`ComplianceCheckResult`](#nemoguardrails-eval-models-ComplianceCheckResult) | Information about a compliance check.                                 |
| [`EvalConfig`](#nemoguardrails-eval-models-EvalConfig)                       | An evaluation configuration for an evaluation dataset.                |
| [`EvalOutput`](#nemoguardrails-eval-models-EvalOutput)                       | The output of the evaluation.                                         |
| [`ExpectedOutput`](#nemoguardrails-eval-models-ExpectedOutput)               | An expected output from the system, as dictated by a policy.          |
| [`GenericOutput`](#nemoguardrails-eval-models-GenericOutput)                 | -                                                                     |
| [`InteractionLog`](#nemoguardrails-eval-models-InteractionLog)               | Detailed log about the execution of an interaction.                   |
| [`InteractionOutput`](#nemoguardrails-eval-models-InteractionOutput)         | The output for running and evaluating an interaction.                 |
| [`InteractionSet`](#nemoguardrails-eval-models-InteractionSet)               | An interaction set description that is part of an evaluation dataset. |
| [`Policy`](#nemoguardrails-eval-models-Policy)                               | A policy for the evaluation of a guardrail configuration.             |
| [`RefusalOutput`](#nemoguardrails-eval-models-RefusalOutput)                 | -                                                                     |
| [`SimilarMessageOutput`](#nemoguardrails-eval-models-SimilarMessageOutput)   | -                                                                     |
| [`Span`](#nemoguardrails-eval-models-Span)                                   | A generic span object                                                 |

### API

```python
class nemoguardrails.eval.models.ComplianceCheckLog()
```

**Bases:** `BaseModel`

Detailed log about a compliance check.

```python
class nemoguardrails.eval.models.ComplianceCheckResult()
```

**Bases:** `BaseModel`

Information about a compliance check.

```python
class nemoguardrails.eval.models.EvalConfig()
```

**Bases:** `BaseModel`

An evaluation configuration for an evaluation dataset.

```python
nemoguardrails.eval.models.EvalConfig.from_path(
    config_path: str
) -> nemoguardrails.eval.models.EvalConfig
```

classmethod

Loads an eval configuration from a given path.

It supports YAML files (\*.yml and *.yaml) and .json files (*.json).

```python
nemoguardrails.eval.models.EvalConfig.validate_policy_ids(
    values: typing.Any
)
```

Validates the policy ids used in the interactions.

```python
class nemoguardrails.eval.models.EvalOutput()
```

**Bases:** `BaseModel`

The output of the evaluation.

```python
nemoguardrails.eval.models.EvalOutput.compute_compliance(
    eval_config: nemoguardrails.eval.models.EvalConfig
) -> typing.Dict[str, dict]
```

Helper to compute the compliance rate per policy.

```python
nemoguardrails.eval.models.EvalOutput.from_path(
    output_path: str
) -> nemoguardrails.eval.models.EvalOutput
```

classmethod

Loads an eval output from a given path.

It supports YAML files (\*.yml and *.yaml) and .json files (*.json).

```python
class nemoguardrails.eval.models.ExpectedOutput()
```

**Bases:** `BaseModel`

An expected output from the system, as dictated by a policy.

```python
class nemoguardrails.eval.models.GenericOutput()
```

**Bases:** [ExpectedOutput](#nemoguardrails-eval-models-ExpectedOutput)

```python
nemoguardrails.eval.models.GenericOutput.__str__()
```

```python
class nemoguardrails.eval.models.InteractionLog()
```

**Bases:** `BaseModel`

Detailed log about the execution of an interaction.

```python
class nemoguardrails.eval.models.InteractionOutput()
```

**Bases:** `BaseModel`

The output for running and evaluating an interaction.

```python
class nemoguardrails.eval.models.InteractionSet()
```

**Bases:** `BaseModel`

An interaction set description that is part of an evaluation dataset.

An interaction set groups multiple interactions with the same expected output.

```python
nemoguardrails.eval.models.InteractionSet.instantiate_expected_output(
    values: typing.Any
)
```

Creates the right instance of the expected output.

```python
class nemoguardrails.eval.models.Policy()
```

**Bases:** `BaseModel`

A policy for the evaluation of a guardrail configuration.

```python
class nemoguardrails.eval.models.RefusalOutput()
```

**Bases:** [ExpectedOutput](#nemoguardrails-eval-models-ExpectedOutput)

```python
nemoguardrails.eval.models.RefusalOutput.__str__()
```

```python
class nemoguardrails.eval.models.SimilarMessageOutput()
```

**Bases:** [ExpectedOutput](#nemoguardrails-eval-models-ExpectedOutput)

```python
nemoguardrails.eval.models.SimilarMessageOutput.__str__()
```

```python
class nemoguardrails.eval.models.Span()
```

**Bases:** `BaseModel`

A generic span object