Run NeMo Guardrails Gateway
Put input and output checks in front of OpenAI Chat Completions and Responses. This guide uses NeMo Guardrails Gateway from the NVIDIA NeMo Guardrails library, native OpenAI bindings, and bundled regex Rails configuration. The regex checks run without a detector model or service; accepted generation requests go to OpenAI.
You need Python 3.11–3.13, uv, and an OpenAI API
key with access to OpenAI GPT-6 Luna.
The examples use gpt-6-luna with one user text input and one assistant text
target. Conversation histories, tools, and multimodal content are outside the
current guarded scope.
Install
Use a library build that includes nemoguardrails.server.experimental and its
server extra. From a source checkout’s repository root:
For an installed library wheel, include [server] when installing that artifact.
Activate its environment, then copy the installed resources outside the checkout:
A separate gateway distribution is not required. An older library release might
not include this module. The server extra includes the OpenAI
Python SDK used in the Python request examples.
Connect to OpenAI
The copied proxy.yaml selects OpenAI and the sibling Rails directory:
Keep proxy.yaml beside rails/config.yml, outside the Rails directory. Relative
Rails paths resolve against the manifest. Omit dialect to use native OpenAI
bindings. Refer to providers and streaming
when connecting a different service; its hostname does not establish its dialect.
The Rails configuration enables input and output regex checks for common email-address patterns. Generation credentials belong in client request headers, outside the manifest. The gateway forwards generation over HTTP; it does not use the library’s model framework to generate answers. Model-backed or service-backed Rails checks have their own dependencies and credentials.
Validate and Start
With the library environment active, run from proxy-example:
validate checks declarations, paths, and routes. inspect describes bindings,
profiles, and limits. Neither initializes Rails or establishes live-provider
compatibility. serve loads every declared Rails configuration before accepting
traffic at http://127.0.0.1:8000.
The launcher supports --host, --port, and --timeout-graceful-shutdown.
Its default request-drain deadline is 30 seconds; resource cleanup follows
draining. It does not authenticate inbound callers. Configure
access controls
before exposing it beyond a trusted local environment.
Send a Buffered Request
Open a second terminal with the same library environment active. Set your own OpenAI API key there:
Choose a shell or Python for each request. Each Python block is a standalone
script; save it and run it with python. The Python client reads
OPENAI_API_KEY and uses max_retries=0 so these examples do not automatically
repeat failed requests. Both clients call the gateway, which forwards the bearer
credential to OpenAI. The bundled Rails checks do not need a server-side key.
Chat Completions
The gateway checks the user text, forwards the accepted request to OpenAI, checks the assistant text, and returns the accepted response. The generated wording varies.
Responses
Responses requires explicit store: false. Omission, true, null, and non-boolean
values are rejected before dispatch. Use one nonempty input string:
Stream Output
Both selected operations also support server-sent event (SSE) streaming. The gateway checks output in windows before release. The bundled settings can hold short answers until completion; refer to streaming windows.
Stream Chat Completions
Stream Responses
Consume the complete stream, including errors and terminal events. A failure after HTTP 200 cannot change that status. The shell displays the SSE events, and the Python SDK can raise an API error during iteration. Earlier approved text cannot be retracted. Refer to stream outcomes.
Verify Input Blocking
The bundled policy rejects this input before sending a generation request to OpenAI:
Expect HTTP 400 and error code content_blocked. The same input rule applies
to Responses. Output checks are also enabled: matching assistant text is
withheld, or its pending stream window is blocked.
Regex checks illustrate configuration and ordering. They do not provide comprehensive protection for personally identifiable information. Choose supported Rails checks for your application. Stop the gateway with Ctrl-C when finished.
Troubleshoot the First Request
For error codes, disconnects, and cleanup warnings, use the troubleshooting guide. Next, configure the gateway, compose a Python application, or inspect its guard contract.