Overview of the NVIDIA NeMo Guardrails Library API Server
The NVIDIA NeMo Guardrails library API server provides the following capabilities:
- Loads guardrails configurations at startup.
- Exposes an OpenAI-compatible REST API for chat completions and model listing.
- Works with the OpenAI Python SDK. Use
OpenAI(base_url="http://localhost:8000/v1"). - Includes a built-in chat UI for testing.
- Supports multiple configurations and combines them for each request.
- Checks messages, tool calls, and tool results against rails without generating a response, through
POST /v1/checks. For more information, refer to Checking Messages Against Rails.
To check the tool calls and tool results of an application that calls its own model, send them to this server’s /v1/checks endpoint.
Tool checks need a configuration that the IORails engine serves, which requires starting the server with NEMO_GUARDRAILS_IORAILS_ENGINE=1.
For an example, refer to Tool Calling: Check Tool Calls and Results Without Generation.
To check a coding agent’s tool calls from its own hooks, use the Guardrails hooks server (nemoguardrails hooks-server) instead.
It is a separate server, with its own command, port, and configuration.
Quick Start
The following steps show how to start the NVIDIA NeMo Guardrails library API server with the provided configuration files and send test requests to the endpoints.
The nemoguardrails server command listens on all network interfaces (0.0.0.0) and does not authenticate incoming API requests.
Restrict network access and place the server behind an authenticated gateway or reverse proxy before exposing it to other users.
Provider API keys authenticate outgoing model calls; they do not protect the Guardrails API.
Prerequisites
Meet the following prerequisites before you use the NVIDIA NeMo Guardrails library API server.
-
Install the NVIDIA NeMo Guardrails library with the
serverextra. For instructions, refer to Extra Dependencies. -
Set the environment variable for your NVIDIA API key.
This key is required to access NVIDIA-hosted models on build.nvidia.com. The provided example configurations and code examples throughout the documentation use NVIDIA-hosted models.
Start the Server
Follow these steps to start the server:
-
Point the server to a parent directory that contains multiple configuration subdirectories:
-
To check if the server is running and list the available configurations, use the following command:
Each subdirectory that contains a config.yml or config.yaml file becomes an available configuration ID.
Send a Request
Send a chat completion request to the server:
View the Chat UI
Open http://localhost:8000 in your browser to access the built-in chat UI for testing.