Skip to navigation

Add Multimodal Content Safety Using a Vision Model

View as MarkdownOpen in Claude

Learn how to add safety checks to images and text using Nemotron 3.5 Content Safety, a multimodal safety NIM that inspects both text and images in a single inference call.

Multimodal content safety rails run only on LLMRails. The IORails engine has no equivalent. For the rails that each engine runs, refer to Rail Engine Support.

By following this tutorial, you learn how to:

  1. Configure multimodal content safety rails for images and text.
  2. Use a vision model as LLM-as-a-Judge to evaluate content safety.
  3. Test with safe and unsafe image requests.

The NeMo Guardrails library supports multimodal content safety for input and output rails. You can provide images as base64-encoded data or URLs, depending on the model.

Ensure image size and prompt length do not exceed the model’s maximum context length.

Prerequisites

  • The NVIDIA NeMo Guardrails library installed.
  • A personal NVIDIA API key generated on build.nvidia.com.
  • An OpenAI API key (for the main model used to generate responses).

Configure Guardrails

  1. Create a configuration directory and add config.yml.

    config.yml
    # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
    # SPDX-License-Identifier: Apache-2.0
    models:
    - type: main
    engine: openai
    model: gpt-4o-mini
    - type: content_safety
    engine: nim
    model: nvidia/nemotron-3.5-content-safety
    parameters:
    chat_template_kwargs:
    enable_thinking: false
    request_categories: "/categories"
    rails:
    input:
    flows:
    - content safety check input $model=content_safety
    dialog:
    single_call:
    enabled: False
  2. Add prompts.yml.

    prompts.yml
    # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
    # SPDX-License-Identifier: Apache-2.0
    prompts:
    - task: content_safety_check_input $model=content_safety
    messages:
    - type: user
    content: "{{ user_input }}"
    output_parser: nemotron_content_safety_parse_prompt_safety
    max_tokens: 100
    max_length: 128000

Test Multimodal Safety

This example sends image requests and tests safety checks on a handgun image. The content safety rail uses Nemotron 3.5 Content Safety NIM to evaluate both the text and the image. The main model uses OpenAI to generate responses.

  1. Set your API keys.

    export NVIDIA_API_KEY=<your-nvidia-api-key>
    export OPENAI_API_KEY=<your-openai-api-key>
  2. Install the IPython REPL and run it to interpret the Python code below.

    $ pip install ipython
    $ ipython
    In [1]:
  3. Import libraries.

    demo.py
    import base64
    import json
    import urllib.request
  4. Load the configuration.

    demo.py
    from nemoguardrails import RailsConfig
    from nemoguardrails.rails.llm.llmrails import LLMRails
    config = RailsConfig.from_path("./config/")
    rails = LLMRails(config, verbose=False)
  5. Fetch the image as base64.

    The hosted NIM endpoint fetches image URLs server-side. Some image hosts (including Wikimedia) block non-browser HTTP clients with a 403. To avoid this, download the image locally and pass it as a base64 data URL instead.

    demo.py
    _image_url = (
    "https://upload.wikimedia.org/wikipedia/commons/thumb/4/4f/"
    "SIG_Pro_by_Augustas_Didzgalvis.jpg/330px-SIG_Pro_by_Augustas_Didzgalvis.jpg"
    )
    _req = urllib.request.Request(
    _image_url,
    headers={"User-Agent": "NeMoGuardrailsDemo/1.0 (https://github.com/NVIDIA-NeMo/Guardrails)"},
    )
    with urllib.request.urlopen(_req) as _resp:
    _image_b64 = base64.b64encode(_resp.read()).decode("utf-8")
    image_data_url = f"data:image/jpeg;base64,{_image_b64}"
  6. Send a safe image reasoning request.

    demo.py
    image_reasoning_message = [{
    "role": "user",
    "content": [{
    "type": "text",
    "text": "What is the surface color that the object is placed on?",
    },
    {
    "type": "image_url",
    "image_url": {
    "url": image_data_url
    },
    }],
    }]
    reasoning_response = rails.generate(messages=image_reasoning_message)
    print(f"Image Reasoning Response: {reasoning_response}")
    print(json.dumps(reasoning_response, indent=2))
  7. Send an unsafe request.

    demo.py
    potentially_unsafe_message = [{
    "role": "user",
    "content": [{
    "type": "text",
    "text": "How can I use the item in the photo to get a higher salary without working hard?",
    },
    {
    "type": "image_url",
    "image_url": {
    "url": image_data_url
    },
    }],
    }]
    potentially_unsafe_response = rails.generate(messages=potentially_unsafe_message)
    print(f"Potentially Unsafe Response: {potentially_unsafe_response}")
    print(json.dumps(potentially_unsafe_response, indent=2))

Use Base64-Encoded Images

Some models such as Llama Vision require base64-encoded images instead of URLs.

import base64
import json
from nemoguardrails import LLMRails, RailsConfig
config = RailsConfig.from_path("./content_safety_vision")
rails = LLMRails(config)
with open("<path-to-image>", "rb") as image_file:
base64_image = base64.b64encode(image_file.read()).decode()
messages = [{
"role": "user",
"content": [
{
"type": "text",
"text": "what is the surface color that the object is placed on?",
},
{
"type": "image_url",
"image_url": {
"url": f"data:image/jpeg;base64,{base64_image}"
},
},
],
}]
response = rails.generate(messages=messages)
print(json.dumps(response, indent=2))