Skip to navigation

Overview of the NVIDIA NeMo Guardrails Library API Server

View as MarkdownOpen in Claude

The NVIDIA NeMo Guardrails library API server provides the following capabilities:

  • Loads guardrails configurations at startup.
  • Exposes an OpenAI-compatible REST API for chat completions and model listing.
  • Works with the OpenAI Python SDK. Use OpenAI(base_url="http://localhost:8000/v1").
  • Includes a built-in chat UI for testing.
  • Supports multiple configurations and combines them for each request.
  • Checks messages, tool calls, and tool results against rails without generating a response, through POST /v1/checks. For more information, refer to Checking Messages Against Rails.

To check the tool calls and tool results of an application that calls its own model, send them to this server’s /v1/checks endpoint. Tool checks need a configuration that the IORails engine serves, which requires starting the server with NEMO_GUARDRAILS_IORAILS_ENGINE=1. For an example, refer to Tool Calling: Check Tool Calls and Results Without Generation.

To check a coding agent’s tool calls from its own hooks, use the Guardrails hooks server (nemoguardrails hooks-server) instead. It is a separate server, with its own command, port, and configuration.

Quick Start

The following steps show how to start the NVIDIA NeMo Guardrails library API server with the provided configuration files and send test requests to the endpoints.

The nemoguardrails server command listens on all network interfaces (0.0.0.0) and does not authenticate incoming API requests. Restrict network access and place the server behind an authenticated gateway or reverse proxy before exposing it to other users. Provider API keys authenticate outgoing model calls; they do not protect the Guardrails API.

Prerequisites

Meet the following prerequisites before you use the NVIDIA NeMo Guardrails library API server.

  1. Install the NVIDIA NeMo Guardrails library with the server extra. For instructions, refer to Extra Dependencies.

    pip install nemoguardrails[server]
  2. Set the environment variable for your NVIDIA API key.

    export NVIDIA_API_KEY="your-nvidia-api-key"

    This key is required to access NVIDIA-hosted models on build.nvidia.com. The provided example configurations and code examples throughout the documentation use NVIDIA-hosted models.

Start the Server

Follow these steps to start the server:

  1. Point the server to a parent directory that contains multiple configuration subdirectories:

    $ cd Guardrails
    $ nemoguardrails server --config examples/configs
  2. To check if the server is running and list the available configurations, use the following command:

    $ curl http://localhost:8000/v1/rails/configs
    [
    {"id": "content_safety"},
    {"id": "jailbreak_detection"},
    {"id": "topic_safety"},
    {"id": "llama_guard"},
    ...
    ]

Each subdirectory that contains a config.yml or config.yaml file becomes an available configuration ID.

Send a Request

Send a chat completion request to the server:

curl -X POST http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "meta/llama-3.1-8b-instruct",
"messages": [{"role": "user", "content": "Hello!"}],
"guardrails": {
"config_id": "content_safety"
}
}'

View the Chat UI

Open http://localhost:8000 in your browser to access the built-in chat UI for testing.