NeMo Guardrails (Deprecated)

Do not start a new deployment on the built-in plugin. Use this documentation to maintain or remove an existing NeMo Relay 0.8 configuration.

The built-in plugin component has kind nemo_guardrails and is available as a first-party NeMo Relay plugin.

The plugin supports these backend modes:

• remote
  • Calls a Guardrails service over HTTP(S), including streaming over the same remote contract.
• local
  • Calls nemoguardrails through a local python3 worker subprocess instead of a separate Guardrails service.

Existing Uses

Existing NeMo Relay 0.8 deployments may use the built-in plugin to:

• Apply Guardrails input and output checks around managed llm.execute(...) calls.
• Apply Guardrails policy around managed tool execution.
• Configure Guardrails behavior through the same plugin config surface used by other first-party NeMo Relay components.
• Keep Guardrails policy authoring in Guardrails-native config while NeMo Relay owns when those checks run around managed execution.

Current Scope

The built-in plugin currently exposes two user-facing modes:

• remote for Guardrails-service integration over HTTP(S)
• local for nemoguardrails integration through a local Python worker

Both modes support managed LLM input and output. The current mode-specific differences are:

• remote supports request_defaults pass-through but does not support managed tool_input
• local supports managed tool_input and broader LLM codec coverage, but it does not support request_defaults

The local backend requires a python3 >= 3.11 executable that can import nemoguardrails==0.22.0. It does not embed Python into the NeMo Relay binary.

Managed Surfaces Versus Request Defaults

Both remote mode and local mode share the same top-level plugin model, but they do not implement every part of that model in the same way.

At the plugin-model level, NeMo Guardrails uses two different concepts:

• Top-level managed NeMo Relay execution surfaces:
  • input
  • output
  • tool_input
  • tool_output
• Guardrails backend request defaults:
  • request_defaults.context
  • request_defaults.thread_id
  • request_defaults.state
  • request_defaults.rails
  • request_defaults.llm_params
  • request_defaults.llm_output
  • request_defaults.output_vars
  • request_defaults.log

This distinction matters:

• Managed surfaces wrap real NeMo Relay execution boundaries such as llm.execute(...) and tools.execute(...).
• Managed surfaces give NeMo Relay an owned enforcement point around a known runtime step. Depending on the backend and surface, Relay can block work, allow it, or apply managed request or result handling before the application sees the outcome.
• Managed surfaces also give NeMo Relay a stable runtime boundary for its own middleware ordering, lifecycle behavior, and observability marks.

The forwarded request-default side is more mode-specific:

• In remote mode, request_defaults fields are forwarded to the selected Guardrails backend as request semantics. They do not create new NeMo Relay-native execution surfaces.
• In local mode, request_defaults is rejected instead of passed through.

The overlap in names is important in remote mode:

• Top-level input is a managed NeMo Relay execution surface.
• request_defaults.rails.input is a backend pass-through option.
• Top-level output is a managed NeMo Relay execution surface.
• request_defaults.rails.output is a backend pass-through option.
• Top-level tool_input is a managed NeMo Relay execution surface in the plugin contract. The current stock-remote backend rejects it, while the local backend supports it.
• request_defaults.rails.tool_input is a backend pass-through option.
• Top-level tool_output is a managed NeMo Relay execution surface.
• request_defaults.rails.tool_output is a backend pass-through option.

In particular, request_defaults.rails.dialog and request_defaults.rails.retrieval are pass-through options. They are not separate managed middleware surfaces in NeMo Relay.

Remote HTTP errors

In remote mode, NeMo Relay preserves received HTTP error status codes from 400 through 599 when returning gateway and lifecycle-hook errors. This applies to buffered requests, streaming setup, and tool input and output checks, even when reading an error response body fails. The existing response envelopes and header handling remain in place. Remote HTTP failures remain non-retryable.

Rust callers receive these failures as FlowError::Upstream with the original status in UpstreamFailure.status, rather than FlowError::Internal. Existing public API signatures and binding exception types remain unchanged, but error messages and classifications change. Explicit guardrail rejections continue to return HTTP 403. Agent-native hook decision responses retain their existing protocol behavior.

The status is the final response received under the existing redirect policy. After streaming response headers have been sent, subsequent failures use the existing stream-error behavior and cannot change the HTTP status.

Pages

• NeMo Guardrails Configuration documents the built-in component shape, mode boundaries, and the detailed support matrix.