> This page is for version v0.25.0.
> For other versions, use one of these documentation indexes:
> - Latest (default): https://docs.nvidia.com/nemo/guardrails/latest/llms.txt
> - v0.25.0: https://docs.nvidia.com/nemo/guardrails/v0.25.0/llms.txt
> - v0.24.1: https://docs.nvidia.com/nemo/guardrails/v0.24.1/llms.txt
> - v0.24.0: https://docs.nvidia.com/nemo/guardrails/v0.24.0/llms.txt
> - v0.23.0: https://docs.nvidia.com/nemo/guardrails/v0.23.0/llms.txt
> - v0.22.0: https://docs.nvidia.com/nemo/guardrails/v0.22.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/guardrails/llms.txt.

# Overview of the NVIDIA NeMo Guardrails Library API Server

> Expose configured guardrails and models through the NVIDIA NeMo Guardrails library HTTP API.

The NVIDIA NeMo Guardrails library API server provides the following capabilities:

* Loads guardrails configurations at startup.
* Exposes an [OpenAI-compatible REST API](https://platform.openai.com/docs/api-reference/chat/create) for chat completions and model listing.
* Works with the [OpenAI Python SDK](https://github.com/openai/openai-python). Use `OpenAI(base_url="http://localhost:8000/v1")`.
* Includes a built-in chat UI for testing.
* Supports multiple configurations and combines them for each request.
* Checks messages, tool calls, and tool results against rails without generating a response, through `POST /v1/checks`.
  For more information, refer to [Checking Messages Against Rails](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-python-apis/check-messages#guardrails-server).

> **Tip**
>
> To check the tool calls and tool results of an application that calls its own model, send them to this server's `/v1/checks` endpoint.
> Tool checks need a configuration that the IORails engine serves, which requires starting the server with `NEMO_GUARDRAILS_IORAILS_ENGINE=1`.
> For an example, refer to [Tool Calling: Check Tool Calls and Results Without Generation](/nemo/guardrails/v0.25.0/configure-guardrails/guardrail-catalog/tool-calling#check-tool-calls-and-results-without-generation).
>
> To check a coding agent's tool calls from its own hooks, use the [Guardrails hooks server](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-hooks-server/overview) (`nemoguardrails hooks-server`) instead.
> It is a separate server, with its own command, port, and configuration.

## Quick Start

The following steps show how to start the NVIDIA NeMo Guardrails library API server with the provided configuration files and send test requests to the endpoints.

> **Warning**
>
> The `nemoguardrails server` command listens on all network interfaces (`0.0.0.0`) and does not authenticate incoming API requests.
> Restrict network access and place the server behind an authenticated gateway or reverse proxy before exposing it to other users.
> Provider API keys authenticate outgoing model calls; they do not protect the Guardrails API.

### Prerequisites

Meet the following prerequisites before you use the NVIDIA NeMo Guardrails library API server.

1. Install the NVIDIA NeMo Guardrails library with the `server` extra. For instructions, refer to [Extra Dependencies](/nemo/guardrails/v0.25.0/get-started/installation-guide#extra-dependencies).

   ```bash
   pip install nemoguardrails[server]
   ```

2. Set the environment variable for your NVIDIA API key.

   ```console
   export NVIDIA_API_KEY="your-nvidia-api-key"
   ```

   This key is required to access NVIDIA-hosted models on [build.nvidia.com](https://build.nvidia.com). The provided [example configurations](https://github.com/NVIDIA-NeMo/Guardrails/tree/develop/examples/configs) and code examples throughout the documentation use NVIDIA-hosted models.

### Start the Server

Follow these steps to start the server:

1. Point the server to a parent directory that contains multiple configuration subdirectories:

   ```console
   $ cd Guardrails
   $ nemoguardrails server --config examples/configs
   ```

2. To check if the server is running and list the available configurations, use the following command:

   ```console
   $ curl http://localhost:8000/v1/rails/configs

   [
     {"id": "content_safety"},
     {"id": "jailbreak_detection"},
     {"id": "topic_safety"},
     {"id": "llama_guard"},
     ...
   ]
   ```

Each subdirectory that contains a `config.yml` or `config.yaml` file becomes an available configuration ID.

### Send a Request

Send a chat completion request to the server:

```bash
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta/llama-3.1-8b-instruct",
    "messages": [{"role": "user", "content": "Hello!"}],
    "guardrails": {
      "config_id": "content_safety"
    }
  }'
```

### View the Chat UI

Open `http://localhost:8000` in your browser to access the built-in chat UI for testing.

## Related Topics

* [Create Chat Completion](https://docs.nvidia.com/nemo/guardrails/v0.25.0/reference/guardrails-api-server/chat-completions/chat-completions)
* [Check Content](https://docs.nvidia.com/nemo/guardrails/v0.25.0/reference/guardrails-api-server/checks/check-content)
* [Introduction to LLM Benchmarking](https://docs.nvidia.com/nim/benchmarking/llm/latest/overview.html)