Skip to navigation

Put input and output checks in front of OpenAI Chat Completions and Responses. This guide uses NeMo Guardrails Gateway from the NVIDIA NeMo Guardrails library, native OpenAI bindings, and bundled regex Rails configuration. The regex checks run without a detector model or service; accepted generation requests go to OpenAI.

You need Python 3.11–3.13, uv, and an OpenAI API key with access to OpenAI GPT-6 Luna. The examples use gpt-6-luna with one user text input and one assistant text target. Conversation histories, tools, and multimodal content are outside the current guarded scope.

Install

Use a library build that includes nemoguardrails.server.experimental and its server extra. From a source checkout’s repository root:

uv sync --locked --extra server
source .venv/bin/activate
cp -R nemoguardrails/server/experimental/examples/basic proxy-example
cd proxy-example

For an installed library wheel, include [server] when installing that artifact. Activate its environment, then copy the installed resources outside the checkout:

python -c 'from importlib.resources import files; from shutil import copytree; copytree(files("nemoguardrails.server.experimental") / "examples/basic", "proxy-example")'
cd proxy-example

A separate gateway distribution is not required. An older library release might not include this module. The server extra includes the OpenAI Python SDK used in the Python request examples.

Connect to OpenAI

The copied proxy.yaml selects OpenAI and the sibling Rails directory:

version: 1
rails_configs:
- id: standard
config_path: ./rails
providers:
- provider: openai
instance_id: openai
base_url: https://api.openai.com/v1
operations: [chat_completions, responses]
rails_config_id: standard

Keep proxy.yaml beside rails/config.yml, outside the Rails directory. Relative Rails paths resolve against the manifest. Omit dialect to use native OpenAI bindings. Refer to providers and streaming when connecting a different service; its hostname does not establish its dialect.

The Rails configuration enables input and output regex checks for common email-address patterns. Generation credentials belong in client request headers, outside the manifest. The gateway forwards generation over HTTP; it does not use the library’s model framework to generate answers. Model-backed or service-backed Rails checks have their own dependencies and credentials.

Validate and Start

With the library environment active, run from proxy-example:

python -m nemoguardrails.server.experimental validate proxy.yaml
python -m nemoguardrails.server.experimental inspect proxy.yaml
python -m nemoguardrails.server.experimental serve proxy.yaml

validate checks declarations, paths, and routes. inspect describes bindings, profiles, and limits. Neither initializes Rails or establishes live-provider compatibility. serve loads every declared Rails configuration before accepting traffic at http://127.0.0.1:8000.

The launcher supports --host, --port, and --timeout-graceful-shutdown. Its default request-drain deadline is 30 seconds; resource cleanup follows draining. It does not authenticate inbound callers. Configure access controls before exposing it beyond a trusted local environment.

Send a Buffered Request

Open a second terminal with the same library environment active. Set your own OpenAI API key there:

export OPENAI_API_KEY="<your OpenAI API key>"

Choose a shell or Python for each request. Each Python block is a standalone script; save it and run it with python. The Python client reads OPENAI_API_KEY and uses max_retries=0 so these examples do not automatically repeat failed requests. Both clients call the gateway, which forwards the bearer credential to OpenAI. The bundled Rails checks do not need a server-side key.

Chat Completions

curl -sS http://127.0.0.1:8000/v1/chat/completions \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{"model":"gpt-6-luna","messages":[{"role":"user","content":"Hello!"}]}'

The gateway checks the user text, forwards the accepted request to OpenAI, checks the assistant text, and returns the accepted response. The generated wording varies.

Responses

Responses requires explicit store: false. Omission, true, null, and non-boolean values are rejected before dispatch. Use one nonempty input string:

curl -sS http://127.0.0.1:8000/v1/responses \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{"model":"gpt-6-luna","input":"Hello!","store":false}'

Stream Output

Both selected operations also support server-sent event (SSE) streaming. The gateway checks output in windows before release. The bundled settings can hold short answers until completion; refer to streaming windows.

Stream Chat Completions

curl -N http://127.0.0.1:8000/v1/chat/completions \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{"model":"gpt-6-luna","messages":[{"role":"user","content":"Hello!"}],"stream":true}'

Stream Responses

curl -N http://127.0.0.1:8000/v1/responses \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{"model":"gpt-6-luna","input":"Hello!","store":false,"stream":true}'

Consume the complete stream, including errors and terminal events. A failure after HTTP 200 cannot change that status. The shell displays the SSE events, and the Python SDK can raise an API error during iteration. Earlier approved text cannot be retracted. Refer to stream outcomes.

Verify Input Blocking

The bundled policy rejects this input before sending a generation request to OpenAI:

curl -sS -i http://127.0.0.1:8000/v1/chat/completions \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{"model":"gpt-6-luna","messages":[{"role":"user","content":"Contact person@example.com"}]}'

Expect HTTP 400 and error code content_blocked. The same input rule applies to Responses. Output checks are also enabled: matching assistant text is withheld, or its pending stream window is blocked.

Regex checks illustrate configuration and ordering. They do not provide comprehensive protection for personally identifiable information. Choose supported Rails checks for your application. Stop the gateway with Ctrl-C when finished.

Troubleshoot the First Request

SymptomCheck
Startup cannot find a Rails directoryconfig_path resolves against the manifest, not your shell directory. Copy the Rails directory with the manifest.
Validation passes but startup failsRails contents, actions, dependencies, and engines load during startup. Declaration validation does not execute them.
OpenAI returns an authentication or model errorCheck OPENAI_API_KEY in the client terminal and access to the model. The gateway does not create credentials or grant model access.
A guarded request or response is rejectedCompare its shape with the selected binding. The provider can support content that the binding does not inspect.
Streaming output arrives lateInspect the Rails window size and any gateway buffering. Final partial windows release only after checking.

For error codes, disconnects, and cleanup warnings, use the troubleshooting guide. Next, configure the gateway, compose a Python application, or inspect its guard contract.