Add Guardrails to an Agent

Guardrails attach to an agent through a guarded virtual model: a VirtualModel entity that uses a guardrail configuration to run input and output rails on every call to the main model. Pointing the agent at the guarded VirtualModel secures the agent’s model path without changing its workflow logic.

There are two steps: create the guarded VirtualModel, then update the agent’s llms block to reference it.

Routing model traffic through rails covers the agent’s model path. It does not cover tool misuse or instructions injected through tool output. To test an agent against those and generate fixes, see Agent Governance.

Prerequisites

Before adding guardrails, make sure you have:

Local services running (nemo services run).
At least one deployed platform-managed agent.
A model provider and model entities registered in the workspace.
A guardrail configuration. See Guardrail Models.

Common catalog models to use as the guardrail backend (verify availability with nemo models list):

nvidia-llama-3-1-nemoguard-8b-content-safety
nvidia-llama-3-1-nemoguard-8b-topic-control
nvidia-llama-3-1-nemotron-safety-guard-8b-v3

1. Create a Guarded VirtualModel

CLI

Skill

Python SDK

nemo inference virtual-models create guarded-agent-model \
  --workspace default \
  --models '[{"model":"default/<main-model-entity>","backend_format":"OPENAI_CHAT"}]' \
  --request-middleware '[{
    "name":"nemo-guardrails",
    "config_type":"guardrail_config",
    "config_id":"default/<guardrail-config>"
  }]' \
  --response-middleware '[{
    "name":"nemo-guardrails",
    "config_type":"guardrail_config",
    "config_id":"default/<guardrail-config>"
  }]'

Wire the same <guardrail-config> on both --request-middleware (for input rails) and --response-middleware (for output rails). Omit a side if the config defines no flows for it. For the full middleware schema, entity-backed versus inline configs, and caching behavior, refer to Guardrails Architecture.

2. Point the Agent at the Guarded VirtualModel

In the agent’s workflow YAML, set model_name on the relevant llms entry to the guarded VirtualModel’s entity reference, with slashes converted to hyphens (per the agent configuration conventions):

llms:
  llm:
    _type: openai
    model_name: default-guarded-agent-model

Leave base_url and api_key unset. Once redeployed, every model call from the agent flows through the guarded VirtualModel. The agent itself is unchanged and unaware of the rails.

For the end-to-end request flow, streaming behavior, header forwarding, and the guardrails request options, refer to Running Inference with Guardrails.

Redeploy the agent, re-run evaluation, and compare quality, cost, latency and safety signals against the baseline before promoting.

Troubleshooting

Virtual model creation fails with an unknown model. Confirm the backend model entity exists with nemo models list. The <main-model-entity> and <guardrail-config> placeholders must reference entities the workspace can resolve.

The agent still calls the unguarded model. Entity references in model_name use hyphens, not slashes. Confirm the agent was redeployed after the config change.

The agents-secure skill is not available. Run nemo skills list to confirm the skill is installed. If it is missing, install it with nemo skills install --agent <claude|codex|cursor|opencode>.