> This page is for version v0.25.0.
> For other versions, use one of these documentation indexes:
> - Latest (default): https://docs.nvidia.com/nemo/guardrails/latest/llms.txt
> - v0.25.0: https://docs.nvidia.com/nemo/guardrails/v0.25.0/llms.txt
> - v0.24.1: https://docs.nvidia.com/nemo/guardrails/v0.24.1/llms.txt
> - v0.24.0: https://docs.nvidia.com/nemo/guardrails/v0.24.0/llms.txt
> - v0.23.0: https://docs.nvidia.com/nemo/guardrails/v0.23.0/llms.txt
> - v0.22.0: https://docs.nvidia.com/nemo/guardrails/v0.22.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/guardrails/llms.txt.

# NeMo Guardrails Gateway

> Apply NeMo Guardrails to existing model APIs with configurable checks that preserve supported provider formats and work with your inference infrastructure.

NeMo Guardrails Gateway provides guardrails enforcement for native model APIs.
Apply the NVIDIA NeMo Guardrails library's configurable checks between your
applications and model services while preserving supported provider formats and
keeping your existing inference infrastructure.

The gateway runs as an HTTP reverse proxy. Provider bindings select the content
to inspect, and Rails supplies the checks. The proxy layer forwards accepted
provider JSON and server-sent event (SSE) bytes without rebuilding them from the
validation models.
Your application or surrounding infrastructure owns access controls, model
routing, budgets, and retry policies.

> **Note**
>
> The gateway currently supports a limited single-text inspection profile. Review
> [provider coverage](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/providers-and-streaming#coverage-of-the-shipped-profiles)
> before connecting your application.
>
> It uses `python -m nemoguardrails.server.experimental`. The [Guardrails API server](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-fastapi-server) (`nemoguardrails server`)
> and [hooks server](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-hooks-server) are separate
> applications with different commands and configuration contracts.

## Start Here

[Run the gateway](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/quickstart), send a request, and verify that input is blocked before
inference using the bundled example. Then choose your next task:

#### [Quickstart](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/quickstart)

Start with OpenAI, then call the gateway from a shell or Python.

#### [Guard Contracts](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/guard-contracts)

Review which content is guarded, constrained, or opaque.

#### [Configuration](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/configuration)

Select upstreams, operation bindings, and Rails configurations.

#### [Python Integration](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/python-integration)

Construct an application or embed it in an ASGI host.

#### [Deployment](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/deployment)

Plan replicas, access controls, readiness, and shutdown.

#### [Troubleshooting](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/troubleshooting)

Diagnose startup, request, streaming, and cleanup failures.

Check [providers and streaming](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/providers-and-streaming)
for accepted shapes and window settings, or read the
[architecture overview](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/architecture).

The architecture section also covers the [request lifecycle](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/architecture/request-lifecycle)
and [framework internals](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/architecture/framework-internals).

## Advanced Configuration

Use these guides for detailed customization, resource ownership, and boundary review:

| Task                                                                     | Guide                                                                                                        |
| ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------ |
| Select Rails from trusted request context                                | [Rails configuration resolution](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/advanced/configuration-resolution) |
| Customize outbound HTTP or route model aliases                           | [HTTP transport customization](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/advanced/http-transport)             |
| Configure several mounts, upstreams, and policies                        | [Multiple provider instances](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/advanced/multiple-provider-instances) |
| Customize request values and response metadata, then export the contract | [Request and response policies](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/advanced/extend-provider-bindings)  |

## How the Pieces Fit

The following components define routing, inspection, and resource ownership:

| Piece                                   | Responsibility                                                                    |
| --------------------------------------- | --------------------------------------------------------------------------------- |
| Application configuration               | Choose upstream instances, operation bindings, dialects, and Rails IDs.           |
| Provider binding and capability profile | Describe which content the current integration and pipeline can handle.           |
| Rails configuration                     | Choose checks for that content and their dependencies.                            |
| Guard contract                          | Make the bound inspection coverage reviewable as an export.                       |
| HTTP transport                          | Perform the outbound exchange, with optional application-specific network policy. |

Check [provider coverage](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/providers-and-streaming#coverage-of-the-shipped-profiles)
before connecting your application. The built-in bindings currently support a
single user text input and one assistant text target; additional content needs
matching binding and checking support.

Start with the [quickstart](/nemo/guardrails/v0.25.0/run-guardrailed-inference/using-gateway/quickstart)
for the library's `server` extra and installed examples. For exact declaration
details, follow the technical references linked from each guide.