Skip to navigation

NeMo Guardrails supports using the ActiveFence ActiveScore API as an input and output rail out-of-the-box (you need to have the ACTIVEFENCE_API_KEY environment variable set).

rails:
input:
flows:
# The simplified version
- activefence moderation on input
# The detailed version with individual risk scores
# - activefence moderation on input detailed

The activefence moderation on input flow blocks when the maximum risk score exceeds 0.7. The activefence moderation on input detailed flow uses individual thresholds per category of violation.

The action returns a RailOutcome whose metadata contains the max_risk_score and the per-category violations scores. To customize the scores, overwrite the default Colang 1.0 flows in your config. For example, to change the threshold for activefence moderation on input to 0.85, add the following flow to your config:

define subflow activefence moderation on input
"""Guardrail based on the maximum risk score."""
$result = execute call_activefence_api(text=$user_message)
if $result.metadata["max_risk_score"] > 0.85
bot refuse to respond
stop

The condition in your flow replaces the built-in decision, so $result.is_blocked is not checked.

ActiveFence’s ActiveScore API gives flexibility in controlling the behavior of various supported violations individually. To leverage that, you can use the violations dictionary in the outcome metadata to set different thresholds for different violations. Below is an example of one such input moderation flow, which you add to rails.input.flows in place of the built-in flow. It checks only the conditions you write, so add one for each category you want to block:

define flow activefence input moderation detailed
$result = execute call_activefence_api(text=$user_message)
if $result.metadata["violations"].get("abusive_or_harmful.hate_speech", 0) > 0.8
bot inform cannot engage in abusive or harmful behavior
stop
define bot inform cannot engage in abusive or harmful behavior
"I will not engage in any abusive or harmful behavior."