Jailbreak Protection
Jailbreak protection helps prevent adversarial attempts from bypassing safety measures and manipulating the LLM into generating harmful or unwanted content.
Jailbreak Detection
The NeMo Guardrails library supports jailbreak detection using a set of heuristics. Currently, two heuristics are supported:
Perplexity is a metric that measures how well a language model predicts text. Lower is better, meaning less randomness or surprise. Typically jailbreak attempts result in higher perplexity.
To activate the jailbreak detection heuristics, you first need include the jailbreak detection heuristics flow as an input rail:
Also, you need to configure the desired thresholds in your config.yml:
If the server_endpoint parameter is not set, the checks will run in-process. This is useful for TESTING PURPOSES ONLY and IS NOT RECOMMENDED FOR PRODUCTION DEPLOYMENTS.
Configuration Validation
The jailbreak detection configuration is validated at create-time. Invalid values raise errors immediately instead of failing silently at runtime. The following validation rules apply:
Heuristics
Length per Perplexity
The length per perplexity heuristic computes the length of the input divided by the perplexity of the input. If the value is above the specified threshold (default 89.79) then the input is considered a jailbreak attempt.
The default value represents the mean length/perplexity for a set of jailbreaks derived from a combination of datasets including AdvBench, ToxicChat, and JailbreakChat, with non-jailbreaks taken from the same datasets and incorporating 1000 examples from Dolly-15k.
The statistics for this metric across jailbreak and non jailbreak datasets are as follows:
Using the mean value of 89.79 yields 31.19% of jailbreaks being detected with a false positive rate of 7.44% on the dataset.
Increasing this threshold will decrease the number of jailbreaks detected but will yield fewer false positives.
USAGE NOTES:
- Manual inspection of false positives uncovered a number of mislabeled examples in the dataset and a substantial number of system-like prompts. If your application is intended for simple question answering or retrieval-aided generation, this should be a generally safe heuristic.
- This heuristic in its current form is intended only for English language evaluation and will yield significantly more false positives on non-English text, including code.
Prefix and Suffix Perplexity
The prefix and suffix perplexity heuristic takes the input and computes the perplexity for the prefix and suffix. If any of the is above the specified threshold (default 1845.65), then the input is considered a jailbreak attempt.
This heuristic examines strings of more than 20 “words” (strings separated by whitespace) to detect potential prefix/suffix attacks.
The default threshold value of 1845.65 is the second-lowest perplexity value across 50 different prompts generated using GCG prefix/suffix attacks.
Using the default value allows for detection of 49/50 GCG-style attacks with a 0.04% false positive rate on the “non-jailbreak” dataset derived above.
USAGE NOTES:
- This heuristic in its current form is intended only for English language evaluation and will yield significantly more false positives on non-English text, including code.
Perplexity Computation
To compute the perplexity of a string, the current implementation uses the gpt2-large model.
Model-based Jailbreak Detections
There is currently one available model-based detection, using a random forest-based detector trained on Snowflake/snowflake-arctic-embed-m-long embeddings.
Input Length Limit
The jailbreak detection model rail accepts up to 2,048 tokens, including special tokens, counted with the Snowflake tokenizer.
It blocks longer inputs before cache lookup, local inference, or a remote detector request.
This limit applies to in-process, server_endpoint, and NIM-backed model checks.
The bundled detector server returns HTTP 413 from /model, which the rail also treats as a block.
This is a breaking behavior change for oversized messages. Earlier versions classified only a truncated prefix and could allow the full message to proceed. The model rail now rejects the message. Applications must keep each submitted message within the detector’s limit, including special tokens.
Input validation uses a bundled, revision-pinned tokenizer and runs offline.
All model checks, including NIM-backed and server_endpoint checks, require the tokenizers package for this validation.
The standard installation currently includes it through FastEmbed. If it is missing from your environment, install it before enabling the model rail:
NIM-backed checks need no Torch, Transformers, Hugging Face credentials, or model downloads.
Configuration loading raises an error with installation instructions if the model rail is enabled and tokenizers is missing.
If the bundled tokenizer cannot be loaded or input coverage cannot be verified at runtime, the rail blocks the request without calling the detector.
Remote requests are sent unchanged only after they pass validation; the model rail also blocks an HTTP 413 returned by the endpoint.
This client-side limit protects calls made through the NVIDIA NeMo Guardrails library. Direct calls to a hosted NIM endpoint depend on that endpoint’s own input handling.
Detector Availability
Jailbreak detection fails closed when a detector cannot complete its check.
Connection failures, timeouts, non-success responses, and malformed results raise errors instead of returning an allow verdict.
A valid response must contain a boolean jailbreak field.
Local model and dependency failures also propagate to engine error handling.
IORails stops processing with a failed blocking outcome and an internal-error response.
LLMRails uses its action-failure handling to return an internal-error response.
These failures do not mean the detector classified the input as a jailbreak.
Applications using check-only APIs must enforce the returned blocked result before continuing.
Outage failures are not cached as detector verdicts.
HTTP 413 responses from model endpoints and inputs beyond the detector’s supported length remain ordinary blocking outcomes, as described above. There is no fail-open configuration option for this rail.
Engine Support
jailbreak detection heuristics shares a rail manifest with jailbreak detection model, and that manifest declares torch and transformers as optional dependencies.
IORails cannot tell from the manifest alone which of the two flows a configuration is using, and so cannot tell whether those packages are actually required.
Rather than guess, it refuses the flow, and a configuration that uses it routes to LLMRails by default.
With require_iorails=True, Guardrails raises ValueError instead of falling back.
This is tracked in issue #2285.
For the full per-rail matrix, refer to Rail Engine Support.
Setup
The recommended way to use the jailbreak detection heuristics in production is to deploy the jailbreak detection heuristics server separately. For a hands-on configuration walkthrough, see Use Jailbreak Detection Heuristics.
For quick testing, you can use the jailbreak detection heuristics rail locally by first installing transformers and torch.
Latency
Latency was tested in-process and via local Docker for both CPU and GPU configurations. For each configuration, we tested the response time for 10 prompts ranging in length from 5 to 2048 tokens. Inference times for sequences longer than the model’s maximum input length (1024 tokens for GPT-2) necessarily take longer. Times reported below in are averages and are reported in milliseconds.