> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/guardrails/llms.txt.
> For full documentation content, see https://docs.nvidia.com/nemo/guardrails/llms-full.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/guardrails/_mcp/server.

# nemoguardrails.evaluate.evaluate_topical

## Module Contents

### Classes

| Name                                                                                         | Description                                                                 |
| -------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------- |
| [`TopicalRailsEvaluation`](#nemoguardrails-evaluate-evaluate_topical-TopicalRailsEvaluation) | Helper class for running the topical rails evaluation for a Guardrails app. |

### Functions

| Name                                                                                                   | Description                                                           |
| ------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------- |
| [`_split_test_set_from_config`](#nemoguardrails-evaluate-evaluate_topical-_split_test_set_from_config) | Extracts a test set of user messages from a config.                   |
| [`cosine_similarity`](#nemoguardrails-evaluate-evaluate_topical-cosine_similarity)                     | Compute the dot product between two embeddings using numpy functions. |
| [`sync_wrapper`](#nemoguardrails-evaluate-evaluate_topical-sync_wrapper)                               | Wrapper for the evaluate\_topical\_rails method which is async.       |

### API

```python
class nemoguardrails.evaluate.evaluate_topical.TopicalRailsEvaluation(
    config: str,
    verbose: typing.Optional[bool] = False,
    test_set_percentage: typing.Optional[float] = 0.3,
    max_tests_per_intent: typing.Optional[int] = 3,
    max_samples_per_intent: typing.Optional[int] = 0,
    print_test_results_frequency: typing.Optional[int] = 10,
    similarity_threshold: typing.Optional[float] = 0.0,
    random_seed: typing.Optional[int] = None,
    output_dir: typing.Optional[str] = None
)
```

Helper class for running the topical rails evaluation for a Guardrails app.
It contains all the configuration parameters required to run the evaluation.

```python
nemoguardrails.evaluate.evaluate_topical.TopicalRailsEvaluation._compute_intent_embeddings(
    intents
)
```

Compute intent embeddings if we have a sentence transformer model.

```python
nemoguardrails.evaluate.evaluate_topical.TopicalRailsEvaluation._get_main_llm_model()
```

```python
nemoguardrails.evaluate.evaluate_topical.TopicalRailsEvaluation._get_most_similar_intent(
    generated_intent
)
```

Retrieves the most similar intent using sentence transformers embeddings.
If the most similar intent is below the similarity threshold,
the generated intent is not changed.

```python
nemoguardrails.evaluate.evaluate_topical.TopicalRailsEvaluation._initialize_embeddings_model()
```

Instantiate a sentence transformer if we use a similarity check for canonical forms.

```python
nemoguardrails.evaluate.evaluate_topical.TopicalRailsEvaluation._initialize_rails_app()
```

```python
nemoguardrails.evaluate.evaluate_topical.TopicalRailsEvaluation._initialize_random_seed()
```

Initialize random seed

```python
nemoguardrails.evaluate.evaluate_topical.TopicalRailsEvaluation._print_evaluation_results(
    processed_samples,
    total_test_samples,
    num_user_intent_errors,
    num_bot_intent_errors,
    num_bot_utterance_errors
)
```

staticmethod

Prints a summary of the evaluation results.

```python
nemoguardrails.evaluate.evaluate_topical.TopicalRailsEvaluation.evaluate_topical_rails()
```

async

Runs the topical evaluation for the Guardrails app with the current configuration.

```python
nemoguardrails.evaluate.evaluate_topical._split_test_set_from_config(
    config: nemoguardrails.RailsConfig,
    test_set_percentage: float,
    test_set: typing.Dict[str, typing.List],
    max_samples_per_intent: int,
    random_seed: typing.Optional[int] = None
)
```

Extracts a test set of user messages from a config.

**Parameters:**

The config from which the test set will be extracted.

The percentage used for the test set.

A dictionary where the test set will be added.

A limit on the number of samples per intent to be enforced.

```python
nemoguardrails.evaluate.evaluate_topical.cosine_similarity(
    v1,
    v2
)
```

Compute the dot product between two embeddings using numpy functions.

```python
nemoguardrails.evaluate.evaluate_topical.sync_wrapper(
    async_func
)
```

Wrapper for the evaluate\_topical\_rails method which is async.