NeMo Guardrails Gateway
NeMo Guardrails Gateway provides guardrails enforcement for native model APIs. Apply the NVIDIA NeMo Guardrails library’s configurable checks between your applications and model services while preserving supported provider formats and keeping your existing inference infrastructure.
The gateway runs as an HTTP reverse proxy. Provider bindings select the content to inspect, and Rails supplies the checks. The proxy layer forwards accepted provider JSON and server-sent event (SSE) bytes without rebuilding them from the validation models. Your application or surrounding infrastructure owns access controls, model routing, budgets, and retry policies.
The gateway currently supports a limited single-text inspection profile. Review provider coverage before connecting your application.
It uses python -m nemoguardrails.server.experimental. The Guardrails API
server (nemoguardrails server)
and hooks server are separate
applications with different commands and configuration contracts.
Start Here
Run the gateway, send a request, and verify that input is blocked before inference using the bundled example. Then choose your next task:
Check providers and streaming for accepted shapes and window settings, or read the architecture overview.
The architecture section also covers the request lifecycle and framework internals.
Advanced Configuration
Use these guides for detailed customization, resource ownership, and boundary review:
How the Pieces Fit
The following components define routing, inspection, and resource ownership:
Check provider coverage before connecting your application. The built-in bindings currently support a single user text input and one assistant text target; additional content needs matching binding and checking support.
Start with the quickstart
for the library’s server extra and installed examples. For exact declaration
details, follow the technical references linked from each guide.