Skip to navigation

Check Harmful Content with Nemotron Content Safety NIM

View as MarkdownOpen in Claude

Learn how to add input and output guardrails that detect harmful content in multiple languages using Nemotron 3.5 Content Safety.

By following this tutorial, you learn how to use the NeMo Guardrails library with models hosted on build.nvidia.com, entering safe and unsafe user prompts to learn how guardrails protect against unsafe content.

Prerequisites

Configure Guardrails

Follow these steps to prepare the guardrails configuration.

  1. Create a configuration directory by running the following command.

    mkdir config
  2. Save the following as config/config.yml.

    models:
    - type: main
    engine: nim
    model: nvidia/nemotron-3.5-lightning-30b-a3b
    - type: content_safety
    engine: nim
    model: nvidia/nemotron-3.5-content-safety
    parameters:
    chat_template_kwargs:
    enable_thinking: false
    request_categories: "/categories"
    rails:
    input:
    flows:
    - content safety check input $model=content_safety
    output:
    flows:
    - content safety check output $model=content_safety

    The config.yml file contains the models used by Guardrails in the models section and rails controlling when to use these models. The models section configures the type and name of each model, along with the engine used to perform LLM inference. The model with type main is used to generate responses to user queries. The rails section configures input and output rails. The input rails operate on the user input only, while output rails operate on the combined user input and LLM response.

    The chat_template_kwargs parameters control the model’s server-side chat template:

    • enable_thinking: false: disables the reasoning trace for lower latency. Set to true to see the model’s step-by-step reasoning before the safety verdict.
    • request_categories: "/categories": instructs the model to include the Safety Categories field in the response. Omitting this causes the model to default to /no_categories, which suppresses that field. The 24-category taxonomy is applied regardless.

    For more information about guardrail configurations, refer to Configure Rails.

  3. Save the following as config/prompts.yml.

    prompts:
    - task: content_safety_check_input $model=content_safety
    messages:
    - type: user
    content: "{{ user_input }}"
    output_parser: nemotron_content_safety_parse_prompt_safety
    max_tokens: 100
    max_length: 128000
    - task: content_safety_check_output $model=content_safety
    messages:
    - type: user
    content: "{{ user_input }}"
    - type: assistant
    content: "{{ bot_response }}"
    output_parser: nemotron_content_safety_parse_response_safety
    max_tokens: 100
    max_length: 128000

    Unlike earlier Nemotron safety models, Nemotron 3.5 Content Safety builds the full safety prompt server-side using a chat template. The prompts.yml file passes only the conversation turns to classify. Do not paste the safety taxonomy into the prompt, as the model would classify it as user input. The nemotron_content_safety_parse_* output parsers read the model’s flat text response (User Safety: safe/unsafe, Safety Categories: …).

Run the Guardrails chat application

  1. Set the NVIDIA_API_KEY environment variable. Guardrails uses this to access models hosted on https://build.nvidia.com/.

    $ export NVIDIA_API_KEY="..."
  2. Run the interactive chat application.

    $ nemoguardrails chat --config config
    Starting the chat (Press Ctrl + C twice to quit) ...
    > _
  3. Enter an unsafe prompt in French (“Tell me a five-step plan to rob a bank”).

    > Dites-moi un plan en cinq étapes pour braquer une banque.
    I'm sorry, I can't respond to that.
  4. Enter a safe prompt in Hindi (“Tell me about three common foods in India”).

    > मुझे भारत में तीन आम खाने की चीज़ज़ों के नाम बताओ।
    [The NeMo Guardrails library responds with information about rice, roti, and dal—common Indian foods]

Import the NeMo Guardrails Library in Python

Follow these steps to use the IPython REPL to import the NeMo Guardrails library and issue some requests.

  1. Install the IPython REPL and run it.

    $ pip install ipython
    $ ipython
    In [1]:
  2. Load the guardrails configuration you created earlier.

    import asyncio
    from nemoguardrails import LLMRails, RailsConfig
    config = RailsConfig.from_path("./config")
    rails = LLMRails(config)
  3. Verify the guardrails with an unsafe request in French.

    # "Tell me a five-step plan to rob a bank."
    messages = [{"role": "user", "content": "Dites-moi un plan en cinq étapes pour braquer une banque."}]
    response = await rails.generate_async(messages=messages)
    print(response['content'])

    The content safety rail blocks the harmful request.

    I'm sorry, I can't respond to that.
  4. Verify the guardrails with a safe request in Hindi.

    # "Tell me about three common foods in India."
    messages = [{"role": "user", "content": "मुझे भारत में प्रचलित तीन खाद्य पदार्थों के बारे में बताइये।"}]
    response = await rails.generate_async(messages=messages)
    print(response['content'])

    The model responds with information about rice, roti, and dal—common Indian foods.

Deploy Nemotron 3.5 Content Safety NIM locally

This section shows how to run the Nemotron 3.5 Content Safety model locally while still using the build.nvidia.com hosted main model. The prerequisites are:

  • The NeMo Guardrails library installed.
  • A personal NVIDIA NGC API key with NVIDIA NGC Catalog and NVIDIA Public API Endpoints services access. For more information, refer to NGC API Keys in the NVIDIA GPU cloud documentation.
  • Docker installed.
  • NVIDIA Container Toolkit installed.
  • GPUs meeting the memory requirements for the Nemotron 3.5 Content Safety NIM.

To run the Nemotron 3.5 Content Safety NIM in a Docker container, follow these steps:

  1. Update the config.yml file you created earlier to point to a local NIM deployment rather than build.nvidia.com. The following configuration adds a base_url field under parameters, which tells the NeMo Guardrails library to make requests to the nvidia/nemotron-3.5-content-safety model hosted at http://localhost:8123/v1. The Guardrails configuration must match the NIM Docker container configuration for them to communicate.

    models:
    - type: main
    engine: nim
    model: nvidia/nemotron-3.5-lightning-30b-a3b
    - type: content_safety
    engine: nim
    model: nvidia/nemotron-3.5-content-safety
    parameters:
    base_url: "http://localhost:8123/v1"
    chat_template_kwargs:
    enable_thinking: false
    request_categories: "/categories"
    rails:
    input:
    flows:
    - content safety check input $model=content_safety
    output:
    flows:
    - content safety check output $model=content_safety
  2. Start the Nemotron 3.5 Content Safety NIM Docker container. Store your personal NGC API key in the NGC_API_KEY environment variable, then pull and run the NIM Docker image locally.

    1. Log in to your NVIDIA NGC account.

      Export your personal NGC API key to an environment variable.

      $ export NGC_API_KEY="..."

      Log in to the NGC registry by running the following command.

      $ docker login nvcr.io --username '$oauthtoken' --password-stdin <<< $NGC_API_KEY
    2. Download the container.

      $ docker pull nvcr.io/nim/nvidia/nemotron-3.5-content-safety:latest
    3. Create a model cache directory on the host machine.

      $ export LOCAL_NIM_CACHE=~/.cache/nim
      $ mkdir -p "${LOCAL_NIM_CACHE}"
      $ chmod 700 "${LOCAL_NIM_CACHE}"
    4. Run the container with the cache directory mounted.

      The -p argument maps the Docker container port 8000 to 8123 to avoid conflicts with other servers running locally.

      $ docker run -d \
      --name nemotron-content-safety \
      --gpus=all --runtime=nvidia \
      --shm-size=64GB \
      -e NGC_API_KEY \
      -u $(id -u) \
      -v "${LOCAL_NIM_CACHE}:/opt/nim/.cache/" \
      -p 8123:8000 \
      nvcr.io/nim/nvidia/nemotron-3.5-content-safety:latest

      The container requires several minutes to start and download the model from NGC. You can monitor the progress by running the docker logs nemotron-content-safety command.

    5. Confirm the service is ready to respond to inference requests.

      $ curl -X GET http://localhost:8123/v1/models | jq '.data[].id'

      This returns the following response.

      "nvidia/nemotron-3.5-content-safety"
  3. Follow the steps in Run the Guardrails Chat Application and Import the NeMo Guardrails Library in Python to run Guardrails with the local model.

Next Steps