NeMo Guardrails Configuration

Use this page when you want to configure the built-in NeMo Guardrails plugin component. The component kind is nemo_guardrails.

The current shipped user-facing backend is mode = "remote". local remains part of the config model, but it is not yet a finished user-facing backend.

For plugin file discovery, precedence, merge behavior, editor controls, and gateway conflict rules, see Plugin Configuration Files.

Component Shape

The top-level NeMo Guardrails object contains:

At least one managed Guardrails surface must be enabled.

Current Remote Support

The current built-in remote backend supports:

Remote Requirements

To use mode = "remote", the configured remote.endpoint must point at a Guardrails service that NeMo Relay can reach from the running process and that exposes the Guardrails remote HTTP(S) contract.

The NeMo Relay plugin config activates Guardrails integration, but the Guardrails service still owns the actual policy content. In practice, NeMo Relay decides when managed checks run, while the Guardrails config decides what to block, allow, or rewrite.

plugins.toml Example

1
version = 1
2
3
[[components]]
4
kind = "nemo_guardrails"
5
enabled = true
6
7
[components.config]
8
version = 1
9
mode = "remote"
10
codec = "openai_chat"
11
input = true
12
output = true
13
tool_output = true
14
15
[components.config.remote]
16
endpoint = "http://127.0.0.1:8000"
17
config_id = "live-smoke"
18
timeout_millis = 3000
19
20
[components.config.request_defaults.context]
21
tenant = "demo"
22
23
[components.config.request_defaults.rails]
24
input = true
25
output = true
26
27
[components.config.policy]
28
unknown_component = "warn"
29
unknown_field = "warn"
30
unsupported_value = "error"

This example configures the built-in remote backend for a Guardrails service that uses codec = "openai_chat", managed LLM input and output, managed tool_output, and request-default pass-through for backend context plus backend input and output rail selection.

In that setup, the NeMo Relay plugin chose the managed surfaces to wrap, while the Guardrails config defined the actual blocking policy, such as rejecting secret-seeking prompts, bypass attempts, specific blocked tokens, or private-key-like output.

For example, the Guardrails-side policy can look like this:

1
rails:
2
  input:
3
    flows:
4
      - self check input
5
  output:
6
    flows:
7
      - self check output

This Guardrails-side config defines the policy logic. The NeMo Relay plugin config decides when those checks run.

Remote Mode Rules

When mode = "remote":

• remote.endpoint is required.
• Exactly one of remote.config_id or remote.config_ids is required.
• config_path, config_yaml, and colang_content cannot be present.
• local settings cannot be present.
• The backend uses the Guardrails remote HTTP(S) contract for both non-streaming and streaming LLM execution.

Codec Boundary

The current built-in remote backend supports managed LLM execution only with:

• openai_chat

Managed Tool Boundary

The current remote backend supports managed tool_output.

The current remote backend rejects managed tool_input explicitly because the stock Guardrails remote contract does not activate pre-execution tool-call rails from externally submitted /v1/chat/completions history. NeMo Relay rejects tool_input in remote mode rather than leaving a silent non-enforcing path.

Request Defaults

request_defaults lets the built-in plugin pass request-time semantics through to the selected backend.

Supported request-default fields are:

• context
• thread_id
• state
• rails
• llm_params
• llm_output
• output_vars
• log

These are backend request options, not additional NeMo Relay-managed execution surfaces.

This includes fields whose names overlap with top-level managed surfaces:

The rails section can include:

• input
• output
• retrieval
• dialog
• tool_output
• tool_input

Those values are forwarded to the backend as request semantics. They do not mean NeMo Relay owns separate managed retrieval or dialog execution surfaces. dialog and retrieval are pass-through request options only. Likewise, request_defaults.rails.tool_input is only a backend pass-through selector. It does not make managed remote tool_input supported in the stock-remote lane.

For more targeted request-time pass-through, the remote backend also forwards selectors like these:

1
[components.config.request_defaults.rails]
2
input = true
3
output = true
4
retrieval = ["retrieve_relevant_chunks"]
5
dialog = true
6
tool_output = ["validate_tool_output"]

This richer selector shape demonstrates how request-time Guardrails semantics can be forwarded even when NeMo Relay does not own a separate native managed surface for that category.

Observability

The current remote backend emits coarse backend-level marks for remote Guardrails activity:

• nemo_guardrails.remote.start
• nemo_guardrails.remote.end
• nemo_guardrails.remote.error

These marks cover managed LLM remote execution and managed tool-result checks.