Skip to navigation

NeMo Guardrails Gateway

View as MarkdownOpen in Claude

NeMo Guardrails Gateway provides guardrails enforcement for native model APIs. Apply the NVIDIA NeMo Guardrails library’s configurable checks between your applications and model services while preserving supported provider formats and keeping your existing inference infrastructure.

The gateway runs as an HTTP reverse proxy. Provider bindings select the content to inspect, and Rails supplies the checks. The proxy layer forwards accepted provider JSON and server-sent event (SSE) bytes without rebuilding them from the validation models. Your application or surrounding infrastructure owns access controls, model routing, budgets, and retry policies.

The gateway currently supports a limited single-text inspection profile. Review provider coverage before connecting your application.

It uses python -m nemoguardrails.server.experimental. The Guardrails API server (nemoguardrails server) and hooks server are separate applications with different commands and configuration contracts.

Start Here

Run the gateway, send a request, and verify that input is blocked before inference using the bundled example. Then choose your next task:

Check providers and streaming for accepted shapes and window settings, or read the architecture overview.

The architecture section also covers the request lifecycle and framework internals.

Advanced Configuration

Use these guides for detailed customization, resource ownership, and boundary review:

TaskGuide
Select Rails from trusted request contextRails configuration resolution
Customize outbound HTTP or route model aliasesHTTP transport customization
Configure several mounts, upstreams, and policiesMultiple provider instances
Customize request values and response metadata, then export the contractRequest and response policies

How the Pieces Fit

The following components define routing, inspection, and resource ownership:

PieceResponsibility
Application configurationChoose upstream instances, operation bindings, dialects, and Rails IDs.
Provider binding and capability profileDescribe which content the current integration and pipeline can handle.
Rails configurationChoose checks for that content and their dependencies.
Guard contractMake the bound inspection coverage reviewable as an export.
HTTP transportPerform the outbound exchange, with optional application-specific network policy.

Check provider coverage before connecting your application. The built-in bindings currently support a single user text input and one assistant text target; additional content needs matching binding and checking support.

Start with the quickstart for the library’s server extra and installed examples. For exact declaration details, follow the technical references linked from each guide.