Overview of the NeMo Guardrails Library API Server#

The NeMo Guardrails API server:

  • Loads guardrails configurations at startup.

  • Exposes a REST API compatible with OpenAI’s chat completions format.

  • Includes a built-in Chat UI for testing.

  • Supports multiple configurations and combining them per-request.

Quick Start#

The following steps show how to start the NeMo Guardrails API server using the provided configuration files and test it by sending requests to the endpoints.

Prerequisites#

Meet the following prerequisites to use the NeMo Guardrails API server.

  1. If you haven’t already, install the NeMo Guardrails library with the nvidia extra, following the instructions in Install the NeMo Guardrails Library.

  2. Set up an environment variable for your NVIDIA API key.

    export NVIDIA_API_KEY="your-nvidia-api-key"

    This is required to access NVIDIA-hosted models on build.nvidia.com. The provided example configurations (examples/configs) and code examples throughout the documentation use NVIDIA-hosted models.

Start the Server#

Point the server to a parent directory containing multiple configuration subdirectories:

$ cd Guardrails
$ nemoguardrails server --config examples/configs

List available configurations:

$ curl http://localhost:8000/v1/rails/configs

[
  {"id": "content_safety"},
  {"id": "jailbreak_detection"},
  {"id": "topic_safety"},
  {"id": "llama_guard"},
  ...
]

Each subdirectory with a config.yml or config.yaml file becomes an available config ID.

Send a Request#

Send a chat completion request to the server:

curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "config_id": "content_safety",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

View the Chat UI#

Open http://localhost:8000 in your browser to access the built-in Chat UI for testing.