DocumentationGuardrail ModelsTutorials

Detecting Injection Attacks with Guardrails

View as Markdown

Detect potential exploitation attempts (such as code injection, cross-site scripting, SQL injection, and template injection) using NeMo Platform.

About Injection Detection

Injection detection is primarily intended for agentic systems as part of a defense-in-depth strategy.

The first part of injection detection is YARA rules. A YARA rule specifies a set of strings (text or binary patterns) to match and a Boolean expression that specifies the rule logic. YARA rules are familiar to many security teams and are easy to audit.

The second part of injection detection is choosing an action when a rule is triggered. You can choose to reject the response and return a refusal such as: “I’m sorry, the desired output triggered rule(s) designed to mitigate exploitation of {detections}.” Rejecting the output is the safest action and most appropriate for production deployments. As an alternative, you can omit the triggering text (masks the offending content).

About the Tutorial

This tutorial demonstrates how to configure basic YARA rules that are part of the NeMo Guardrails toolkit. You can view the default rules in the yara_rules directory. The default rules support SQL injection, cross-site scripting (XSS), Jinja template injection, and Python code that uses shells, networking, and more.

For the main model, this tutorial uses the Llama-3.1-8B-Instruct NIM.

Prerequisites

Before you begin:

  • You have access to a running NeMo Platform.
  • NMP_BASE_URL is set to the NeMo Platform base URL.
  • A ModelProvider is configured with an LLM provider. Follow Setup if you haven’t done this yet.

This tutorial uses the following NIM, available on build.nvidia.com:

  • main model: meta/llama-3.1-8b-instruct

Step 1: Configure the Client

Instantiate the platform client.

1import os
2from nemo_platform import NeMoPlatform, ConflictError
3
4client = NeMoPlatform(
5    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
6    workspace="default",
7)

Step 2: Create a Guardrail Configuration

This config enables injection detection and applies it to model output.

1guardrails_config = {
2    "rails": {
3        "config": {
4            "injection_detection": {
5                "injections": ["code", "sqli", "template", "xss"],
6                "action": "reject",
7            }
8        },
9        "output": {"flows": ["injection detection"]},
10    },
11}
12
13config_name = "injection-detection-config"
14try:
15    config = client.guardrail.configs.create(
16        name=config_name,
17        description="Injection detection guardrails",
18        data=guardrails_config,
19    )
20except ConflictError:
21    print(f"Config {config_name} already exists, continuing...")

The rails.config.injection_detection field configures how to apply the injection detection rules. It supports the following fields:

FieldTypeRequiredDescription
injectionslist of stringsYesList of injection categories to detect (e.g., "code", "sqli", "template", "xss")
actionstringYesAction to take when an injection is detected: "reject" (block and return refusal) or "omit" (mask the offending content)
yara_rulesobjectNoCustom YARA rules as key-value pairs, where the key is the rule name and the value is the YARA rule definition

Step 3: Create a VirtualModel

Create a VirtualModel that routes inference through the guardrails middleware. Since injection detection uses output rails only, only response_middleware is needed.

$nemo inference virtual-models create guarded-injection-detect \
>  --default-model-entity default/meta-llama-3-1-8b-instruct \
>  --response-middleware '[{"name":"nemo-guardrails","config_type":"guardrail_config","config_id":"default/injection-detection-config"}]'

Step 4: Verify Blocked Content

Get a pre-configured OpenAI client from the SDK and send a request for Python code that uses networking packages, which is likely to trigger injection detection:

1oai_client = client.models.get_openai_client()
2
3response = oai_client.chat.completions.create(
4    model="default/guarded-injection-detect",
5    messages=[
6        {
7            "role": "user",
8            "content": "Write a Python script that uses requests and urllib to fetch weather data.",
9        }
10    ],
11    max_tokens=200,
12)
13
14print(response.model_dump_json(indent=2))
1{
2  "id": "chatcmpl-6e6ee35f-87be-4372-8f3d-f4f0c61f51db",
3  "object": "chat.completion",
4  "model": "meta/llama-3.1-8b-instruct",
5  "choices": [
6    {
7      "index": 0,
8      "message": {
9        "role": "assistant",
10        "content": "I'm sorry, the desired output triggered rule(s) designed to mitigate exploitation of code."
11      },
12      "finish_reason": "content_filter"
13    }
14  ]
15}

Step 5: Verify Allowed Content

Send a safe request and confirm you receive a normal response:

1response = oai_client.chat.completions.create(
2    model="default/guarded-injection-detect",
3    messages=[
4        {
5            "role": "user",
6            "content": "Tell me about Cape Hatteras National Seashore in 50 words or less.",
7        }
8    ],
9    max_tokens=100,
10)
11
12print(response.model_dump_json(indent=2))
1{
2  "id": "chatcmpl-3f3f3d2e-2caa-4f89-9a46-8c2b2d0b1f8c",
3  "object": "chat.completion",
4  "model": "meta/llama-3.1-8b-instruct",
5  "choices": [
6    {
7      "index": 0,
8      "message": {
9        "role": "assistant",
10        "content": "Cape Hatteras National Seashore protects barrier islands, beaches, and lighthouses along North Carolina's Outer Banks."
11      },
12      "finish_reason": "stop"
13    }
14  ]
15}

Optional: Specify Inline Rules

Provide custom YARA rules inline. The example below performs a case-insensitive check for the word “Ethernet” and rejects the response if it appears.

1inline_rules_config = client.guardrail.configs.create(
2    name="injection-detection-inline-config",
3    description="Injection detection with inline YARA rules",
4    data={
5        "rails": {
6            "config": {
7                "injection_detection": {
8                    "injections": ["reject_ethernet"],
9                    "yara_rules": {
10                        "reject_ethernet": 'rule reject_ethernet {\n strings:\n $string = "ethernet" nocase\n condition:\n $string\n}'
11                    },
12                    "action": "reject",
13                }
14            },
15            "output": {"flows": ["injection detection"]},
16        },
17    },
18)

Create a VirtualModel for the inline config:

1client.inference.virtual_models.create(
2    name="guarded-injection-inline",
3    default_model_entity="default/meta-llama-3-1-8b-instruct",
4    response_middleware=[
5        {
6            "name": "nemo-guardrails",
7            "config_type": "guardrail_config",
8            "config_id": "default/injection-detection-inline-config",
9        }
10    ],
11)

Send a request that contains the word “ethernet”, which triggers the rule:

1response = oai_client.chat.completions.create(
2    model="default/guarded-injection-inline",
3    messages=[{"role": "user", "content": "Explain Ethernet headers."}],
4    max_tokens=100,
5)
6
7print(response.model_dump_json(indent=2))
1{
2  "id": "chatcmpl-9b2c6b21-7f5f-4a3c-9c77-2a4b2e4b6b2a",
3  "object": "chat.completion",
4  "model": "meta/llama-3.1-8b-instruct",
5  "choices": [
6    {
7      "index": 0,
8      "message": {
9        "role": "assistant",
10        "content": "I'm sorry, the desired output triggered rule(s) designed to mitigate exploitation of reject_ethernet."
11      },
12      "finish_reason": "content_filter"
13    }
14  ]
15}

Cleanup

1client.inference.virtual_models.delete(name="guarded-injection-detect")
2client.guardrail.configs.delete(name=config_name)
3# Uncomment if Optional section was run:
4# client.inference.virtual_models.delete(name="guarded-injection-inline")
5# client.guardrail.configs.delete(name="injection-detection-inline-config")
6print("Cleanup complete")