> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/guardrails/llms.txt.
> For full documentation content, see https://docs.nvidia.com/nemo/guardrails/llms-full.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/guardrails/_mcp/server.

# nemoguardrails.server.api

## Module Contents

### Classes

| Name                                                                                      | Description                                                               |
| ----------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- |
| [`ChunkError`](#nemoguardrails-server-api-ChunkError)                                     | -                                                                         |
| [`ChunkErrorMetadata`](#nemoguardrails-server-api-ChunkErrorMetadata)                     | -                                                                         |
| [`GuardrailsApp`](#nemoguardrails-server-api-GuardrailsApp)                               | Custom FastAPI subclass with additional attributes for Guardrails server. |
| [`GuardrailsConfigurationError`](#nemoguardrails-server-api-GuardrailsConfigurationError) | Exception raised for errors in the configuration.                         |

### Functions

| Name                                                                                      | Description                                                                     |
| ----------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
| [`_format_streaming_response`](#nemoguardrails-server-api-_format_streaming_response)     | Format streaming chunks from LLMRails.stream\_async() as SSE events.            |
| [`_generate_cache_key`](#nemoguardrails-server-api-_generate_cache_key)                   | Generates a cache key for the given config ids and model name.                  |
| [`_get_rails`](#nemoguardrails-server-api-_get_rails)                                     | Returns the rails instance for the given config id and model.                   |
| [`_has_config_file`](#nemoguardrails-server-api-_has_config_file)                         | Check if a directory (or its 'config' subdirectory) contains a config.yml/yaml. |
| [`_raise_invalid_state`](#nemoguardrails-server-api-_raise_invalid_state)                 | -                                                                               |
| [`_update_models_in_config`](#nemoguardrails-server-api-_update_models_in_config)         | Update the main model in the RailsConfig.                                       |
| [`_validate_public_state_shape`](#nemoguardrails-server-api-_validate_public_state_shape) | Validate request state shape before loading rails config.                       |
| [`chat_completion`](#nemoguardrails-server-api-chat_completion)                           | Chat completion for the provided conversation.                                  |
| [`get_challenges`](#nemoguardrails-server-api-get_challenges)                             | Returns the list of available challenges for red teaming.                       |
| [`get_rails_configs`](#nemoguardrails-server-api-get_rails_configs)                       | Returns the list of available rails configurations.                             |
| [`lifespan`](#nemoguardrails-server-api-lifespan)                                         | Register any additional challenges, if available at startup.                    |
| [`list_models`](#nemoguardrails-server-api-list_models)                                   | Return the list of models available from the configured provider.               |
| [`process_chunk`](#nemoguardrails-server-api-process_chunk)                               | Processes a single chunk from the stream.                                       |
| [`register_challenges`](#nemoguardrails-server-api-register_challenges)                   | Register additional challenges                                                  |
| [`register_datastore`](#nemoguardrails-server-api-register_datastore)                     | Registers a DataStore to be used by the server.                                 |
| [`register_logger`](#nemoguardrails-server-api-register_logger)                           | Register an additional logger                                                   |
| [`root_handler`](#nemoguardrails-server-api-root_handler)                                 | -                                                                               |
| [`root_redirect`](#nemoguardrails-server-api-root_redirect)                               | -                                                                               |
| [`set_default_config_id`](#nemoguardrails-server-api-set_default_config_id)               | -                                                                               |
| [`start_auto_reload_monitoring`](#nemoguardrails-server-api-start_auto_reload_monitoring) | Start a thread that monitors the config folder for changes.                     |

### Data

[`ALLOWED_ORIGINS`](#nemoguardrails-server-api-ALLOWED_ORIGINS)

[`ENABLE_CORS`](#nemoguardrails-server-api-ENABLE_CORS)

[`api_description`](#nemoguardrails-server-api-api_description)

[`api_request_headers`](#nemoguardrails-server-api-api_request_headers)

[`app`](#nemoguardrails-server-api-app)

[`chainlit_app_path`](#nemoguardrails-server-api-chainlit_app_path)

[`challenges`](#nemoguardrails-server-api-challenges)

[`datastore`](#nemoguardrails-server-api-datastore)

[`llm_rails_events_history_cache`](#nemoguardrails-server-api-llm_rails_events_history_cache)

[`llm_rails_instances`](#nemoguardrails-server-api-llm_rails_instances)

[`log`](#nemoguardrails-server-api-log)

[`origins`](#nemoguardrails-server-api-origins)

[`registered_loggers`](#nemoguardrails-server-api-registered_loggers)

### API

```python
class nemoguardrails.server.api.ChunkError()
```

**Bases:** `BaseModel`

```python
class nemoguardrails.server.api.ChunkErrorMetadata()
```

**Bases:** `BaseModel`

```python
class nemoguardrails.server.api.GuardrailsApp(
    args = (),
    kwargs = {}
)
```

**Bases:** `FastAPI`

Custom FastAPI subclass with additional attributes for Guardrails server.

```python
class nemoguardrails.server.api.GuardrailsConfigurationError()
```

Exception

**Bases:** `Exception`

Exception raised for errors in the configuration.

```python
nemoguardrails.server.api._format_streaming_response(
    stream_iterator: typing.AsyncIterator[typing.Union[str, dict]],
    model_name: str
) -> typing.AsyncIterator[str]
```

async

Format streaming chunks from LLMRails.stream\_async() as SSE events.

Args:
stream\_iterator: AsyncIterator from stream\_async() that yields str or dict chunks
model\_name: The model name to include in the chunks

Yields:
SSE-formatted strings (data: \{...}

)

```python
nemoguardrails.server.api._generate_cache_key(
    config_ids: typing.List[str],
    model_name: typing.Optional[str] = None
) -> str
```

Generates a cache key for the given config ids and model name.

```python
nemoguardrails.server.api._get_rails(
    config_ids: typing.List[str],
    model_name: typing.Optional[str] = None
) -> nemoguardrails.LLMRails
```

async

Returns the rails instance for the given config id and model.

**Parameters:**

List of configuration IDs to load

The model name from the request (overrides config's main model)

```python
nemoguardrails.server.api._has_config_file(
    path: str
) -> bool
```

Check if a directory (or its 'config' subdirectory) contains a config.yml/yaml.

```python
nemoguardrails.server.api._raise_invalid_state(
    detail: str
) -> None
```

```python
nemoguardrails.server.api._update_models_in_config(
    config: nemoguardrails.RailsConfig,
    main_model: nemoguardrails.rails.llm.config.Model
) -> nemoguardrails.RailsConfig
```

Update the main model in the RailsConfig.

If a model with type="main" exists, it replaces it. Otherwise, adds it.

```python
nemoguardrails.server.api._validate_public_state_shape(
    state: typing.Optional[dict]
) -> None
```

Validate request state shape before loading rails config.

At the public HTTP boundary, the only accepted non-empty dict state shape is
Colang 1.0 transcript state: \{"events": \[...]}. Colang 2.0 has no safe
public dict state shape.

```python
nemoguardrails.server.api.chat_completion(
    body: nemoguardrails.server.schemas.openai.GuardrailsChatCompletionRequest,
    request: fastapi.Request
)
```

async

Chat completion for the provided conversation.

TODO: add support for explicit state object.

```python
nemoguardrails.server.api.get_challenges()
```

async

Returns the list of available challenges for red teaming.

```python
nemoguardrails.server.api.get_rails_configs()
```

async

Returns the list of available rails configurations.

```python
nemoguardrails.server.api.lifespan(
    app: nemoguardrails.server.api.GuardrailsApp
)
```

async

Register any additional challenges, if available at startup.

```python
nemoguardrails.server.api.list_models(
    request: fastapi.Request
)
```

async

Return the list of models available from the configured provider.

```python
nemoguardrails.server.api.process_chunk(
    chunk: typing.Any
) -> typing.Union[typing.Any, nemoguardrails.server.api.ChunkError]
```

Processes a single chunk from the stream.

**Parameters:**

A single chunk from the stream (can be str, dict, or other type).

The model name (not used in processing but kept for signature consistency).

**Returns:** `Union[Any, ChunkError]`

Union\[Any, StreamingError]: StreamingError instance for errors or the original chunk.

```python
nemoguardrails.server.api.register_challenges(
    additional_challenges: typing.List[dict]
)
```

Register additional challenges

**Parameters:**

The new challenges to be registered.

```python
nemoguardrails.server.api.register_datastore(
    datastore_instance: nemoguardrails.server.datastore.datastore.DataStore
)
```

Registers a DataStore to be used by the server.

```python
nemoguardrails.server.api.register_logger(
    logger: typing.Callable
)
```

Register an additional logger

```python
nemoguardrails.server.api.root_handler()
```

async

```python
nemoguardrails.server.api.root_redirect()
```

async

```python
nemoguardrails.server.api.set_default_config_id(
    config_id: str
)
```

```python
nemoguardrails.server.api.start_auto_reload_monitoring()
```

Start a thread that monitors the config folder for changes.

```python
nemoguardrails.server.api.ALLOWED_ORIGINS = os.getenv('NEMO_GUARDRAILS_SERVER_ALLOWED_ORIGINS', '*')
```

```python
nemoguardrails.server.api.ENABLE_CORS = os.getenv('NEMO_GUARDRAILS_SERVER_ENABLE_CORS', 'false').lower() == 'true'
```

```python
nemoguardrails.server.api.api_description = 'Guardrails Server API.'
```

```python
nemoguardrails.server.api.api_request_headers: ContextVar = contextvars.ContextVar('headers')
```

```python
nemoguardrails.server.api.app = GuardrailsApp(title='Guardrails Server API', description=api_description, versio...
```

```python
nemoguardrails.server.api.chainlit_app_path = os.path.join(os.path.dirname(__file__), 'app.py')
```

```python
nemoguardrails.server.api.challenges = []
```

```python
nemoguardrails.server.api.datastore: Optional[DataStore] = None
```

```python
nemoguardrails.server.api.llm_rails_events_history_cache: dict[str, dict] = {}
```

```python
nemoguardrails.server.api.llm_rails_instances: dict[str, LLMRails] = {}
```

```python
nemoguardrails.server.api.log = logging.getLogger(__name__)
```

```python
nemoguardrails.server.api.origins = ALLOWED_ORIGINS.split(',')
```

```python
nemoguardrails.server.api.registered_loggers: List[Callable] = []
```