Skip to navigation

NeMo Guardrails supports using the GCP Text Moderation API as an input rail out-of-the-box. There are many ways you can authentication on GCP, refer to this link for more details .

rails:
input:
flows:
# The simplified version
- gcpnlp moderation
# The detailed version with individual risk scores
# - gcpnlp moderation detailed

The gcpnlp moderation flow uses the maximum risk score with an 0.80 threshold to decide if the input should be allowed or not (i.e., if the risk score is above the threshold, it is considered a violation). The gcpnlp moderation detailed has individual scores per category of violation.

The action returns a RailOutcome whose metadata contains the max_risk_score and the per-category violations scores. To customize the scores, overwrite the default Colang 1.0 flows in your config. For example, to change the threshold for gcpnlp moderation to 0.9, add the following flow to your config:

define subflow gcpnlp moderation
"""Guardrail based on the maximum risk score."""
$result = execute call_gcpnlp_api
if $result.metadata["max_risk_score"] > 0.9
bot refuse to respond
stop

The condition in your flow replaces the built-in decision, so $result.is_blocked is not checked.

Using GCP Text Moderation, you can control various violations individually. The violations dictionary in the outcome metadata maps each moderation category that GCP returns to its confidence score. Below is an example of one such input moderation flow, which you add to rails.input.flows in place of the built-in flow. It checks only the conditions you write, so add one for each category you want to block:

define flow gcpnlp input moderation detailed
$result = execute call_gcpnlp_api(user_message=$user_message)
if $result.metadata["violations"].get("Toxic", 0) > 0.8
bot inform cannot engage in abusive or harmful behavior
stop
define bot inform cannot engage in abusive or harmful behavior
"I will not engage in any abusive or harmful behavior."