> This page is for NeMo Relay, version v0.10.1. > For other versions, use one of these documentation indexes: > - Latest (v0.10.1) (default): https://docs.nvidia.com/nemo/relay/latest/llms.txt > - dev: https://docs.nvidia.com/nemo/relay/dev/llms.txt > - v0.10.1: https://docs.nvidia.com/nemo/relay/v0.10.1/llms.txt > - v0.9.4: https://docs.nvidia.com/nemo/relay/v0.9.4/llms.txt > - v0.8.4: https://docs.nvidia.com/nemo/relay/v0.8.4/llms.txt > - v0.7.3: https://docs.nvidia.com/nemo/relay/v0.7.3/llms.txt > - v0.6.0: https://docs.nvidia.com/nemo/relay/v0.6.0/llms.txt > - v0.5.0: https://docs.nvidia.com/nemo/relay/v0.5.0/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo/relay/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/relay/_mcp/server. # NeMo Guardrails (Deprecated) > Review the deprecated built-in NeMo Guardrails plugin for managed LLM and tool execution. > **Deprecated** > > The built-in `nemo_guardrails` plugin is deprecated and scheduled for removal > in NeMo Relay 0.9. It remains available in NeMo Relay 0.8. There is no > replacement in NeMo Relay 0.8; any replacement will target 0.9 or later. Do not start a new deployment on the built-in plugin. Use this documentation to maintain or remove an existing NeMo Relay 0.8 configuration. The built-in plugin component has kind `nemo_guardrails` and is available as a first-party NeMo Relay plugin. The plugin supports these backend modes: * `remote` * Calls a Guardrails service over HTTP(S), including streaming over the same remote contract. * `local` * Calls `nemoguardrails` through a local `python3` worker subprocess instead of a separate Guardrails service. ## Existing Uses Existing NeMo Relay 0.8 deployments may use the built-in plugin to: * Apply Guardrails input and output checks around managed `llm.execute(...)` calls. * Apply Guardrails policy around managed tool execution. * Configure Guardrails behavior through the same plugin config surface used by other first-party NeMo Relay components. * Keep Guardrails policy authoring in Guardrails-native config while NeMo Relay owns when those checks run around managed execution. ## Current Scope The built-in plugin currently exposes two user-facing modes: * `remote` for Guardrails-service integration over HTTP(S) * `local` for `nemoguardrails` integration through a local Python worker Both modes support managed LLM `input` and `output`. The current mode-specific differences are: * `remote` supports `request_defaults` pass-through but does not support managed `tool_input` * `local` supports managed `tool_input` and broader LLM codec coverage, but it does not support `request_defaults` The `local` backend requires a `python3 >= 3.11` executable that can import `nemoguardrails==0.22.0`. It does not embed Python into the NeMo Relay binary. ## Managed Surfaces Versus Request Defaults Both `remote` mode and `local` mode share the same top-level plugin model, but they do not implement every part of that model in the same way. At the plugin-model level, NeMo Guardrails uses two different concepts: * Top-level managed NeMo Relay execution surfaces: * `input` * `output` * `tool_input` * `tool_output` * Guardrails backend request defaults: * `request_defaults.context` * `request_defaults.thread_id` * `request_defaults.state` * `request_defaults.rails` * `request_defaults.llm_params` * `request_defaults.llm_output` * `request_defaults.output_vars` * `request_defaults.log` This distinction matters: * Managed surfaces wrap real NeMo Relay execution boundaries such as `llm.execute(...)` and `tools.execute(...)`. * Managed surfaces give NeMo Relay an owned enforcement point around a known runtime step. Depending on the backend and surface, Relay can block work, allow it, or apply managed request or result handling before the application sees the outcome. * Managed surfaces also give NeMo Relay a stable runtime boundary for its own middleware ordering, lifecycle behavior, and observability marks. The forwarded request-default side is more mode-specific: * In `remote` mode, `request_defaults` fields are forwarded to the selected Guardrails backend as request semantics. They do not create new NeMo Relay-native execution surfaces. * In `local` mode, `request_defaults` is rejected instead of passed through. The overlap in names is important in `remote` mode: * Top-level `input` is a managed NeMo Relay execution surface. * `request_defaults.rails.input` is a backend pass-through option. * Top-level `output` is a managed NeMo Relay execution surface. * `request_defaults.rails.output` is a backend pass-through option. * Top-level `tool_input` is a managed NeMo Relay execution surface in the plugin contract. The current stock-remote backend rejects it, while the local backend supports it. * `request_defaults.rails.tool_input` is a backend pass-through option. * Top-level `tool_output` is a managed NeMo Relay execution surface. * `request_defaults.rails.tool_output` is a backend pass-through option. In particular, `request_defaults.rails.dialog` and `request_defaults.rails.retrieval` are pass-through options. They are not separate managed middleware surfaces in NeMo Relay. ## Remote HTTP errors In remote mode, NeMo Relay preserves received HTTP error status codes from 400 through 599 when returning gateway and lifecycle-hook errors. This applies to buffered requests, streaming setup, and tool input and output checks, even when reading an error response body fails. The existing response envelopes and header handling remain in place. Remote HTTP failures remain non-retryable. Rust callers receive these failures as `FlowError::Upstream` with the original status in `UpstreamFailure.status`, rather than `FlowError::Internal`. Existing public API signatures and binding exception types remain unchanged, but error messages and classifications change. Explicit guardrail rejections continue to return HTTP 403. Agent-native hook decision responses retain their existing protocol behavior. The status is the final response received under the existing redirect policy. After streaming response headers have been sent, subsequent failures use the existing stream-error behavior and cannot change the HTTP status. ## Pages * [NeMo Guardrails Configuration](/configure-plugins/nemo-guardrails/configuration) documents the built-in component shape, mode boundaries, and the detailed support matrix.