Skip to navigation

Custom Policy with Nemotron 3.5 Content Safety NIM

View as MarkdownOpen in Claude

Learn how to restrict conversations to allowed topics using a custom policy with Nemotron 3.5 Content Safety.

A custom policy lets you define your own safety rules for a specific use case, for example limiting a customer service bot to company-related topics. The Nemotron 3.5 Content Safety NIM supports two mutually exclusive custom-field modes: custom_policy (Mode A) accepts free-text behavioral instructions and replaces the built-in safety behavior entirely; custom_taxonomy (Mode B) accepts a structured list of harm categories that replaces the built-in 24-category taxonomy. This tutorial uses custom_policy. To apply both the built-in taxonomy and a custom policy simultaneously, use two separate model entries in config.yml: one with request_categories: "/categories" for the built-in taxonomy and one with custom_policy for your rules.

By following this tutorial, you learn how to configure a custom policy and interact with both on-topic and off-topic requests.

Prerequisites

Configure Guardrails

  1. Create a configuration directory.

    mkdir config
  2. Save the following as config/config.yml.

    models:
    - type: main
    engine: nim
    model: nvidia/nemotron-3.5-lightning-30b-a3b
    - type: content_safety
    engine: nim
    model: nvidia/nemotron-3.5-content-safety
    parameters:
    chat_template_kwargs:
    enable_thinking: false
    request_categories: "/categories"
    custom_policy: |
    You are to act as a customer service agent, providing users with factual information in accordance with the knowledge base. Your role is to ensure that you respond only to relevant queries and adhere to the following guidelines:
    - Do not answer questions related to personal opinions or advice on a user's order or future recommendations.
    - Do not provide any information on non-company products or services.
    - Do not answer enquiries unrelated to the company policies.
    - Do not answer questions asking for personal details about the agent or its creators.
    - Do not answer questions about sensitive topics related to politics, religion, or other sensitive subjects.
    - If a user asks about topics irrelevant to the company's customer service, politely redirect the conversation or end the interaction.
    - Allow user comments that are related to small talk and chit-chat.
    rails:
    input:
    flows:
    - content safety check input $model=content_safety

    The custom_policy field is passed to the model’s chat template as free-text behavioral instructions (Mode A). The model applies your policy and returns User Safety: safe or User Safety: unsafe. Topic control only operates on user input, so there is no output rail flow.

    For more information about guardrail configurations, refer to Configure Rails.

  3. Save the following as config/prompts.yml.

    prompts:
    - task: content_safety_check_input $model=content_safety
    messages:
    - type: user
    content: "{{ user_input }}"
    output_parser: nemotron_content_safety_parse_prompt_safety
    max_tokens: 100
    max_length: 128000

    Nemotron 3.5 Content Safety builds the full safety prompt server-side, including your custom_policy. The prompts.yml passes only the user turn to classify. The nemotron_content_safety_parse_prompt_safety output parser reads the User Safety: safe/unsafe verdict line.

Run the Guardrails Chat Application

  1. Set the NVIDIA_API_KEY environment variable.

    $ export NVIDIA_API_KEY="..."
  2. Run the interactive chat application.

    $ nemoguardrails chat --config config
    Starting the chat (Press Ctrl + C twice to quit) ...
    > _
  3. Enter an off-topic request.

    The custom policy prohibits questions about politics. The content safety input rail detects a policy violation and returns the refusal text. The main model is never called.

    > Which party should I vote for in the next election?
    I'm sorry, I can't respond to that.
  4. Enter an on-topic request.

    This request is within the allowed topics, so the content safety rail passes it through to the main model.

    > I'd like to cancel my subscription. Can I do this by phone or on the website?
    I'd be happy to help you with canceling your subscription. You have a couple of options to do so...
    [The NeMo Guardrails library responds with instructions and information on subscription cancellations]

Import the NeMo Guardrails Library in Python

  1. Install the IPython REPL and run it.

    $ pip install ipython
    $ ipython
    In [1]:
  2. Load the guardrails configuration you created earlier.

    import asyncio
    from nemoguardrails import LLMRails, RailsConfig
    config = RailsConfig.from_path("./config")
    rails = LLMRails(config)
  3. Verify the guardrails with an off-topic political question.

    messages = [{"role": "user", "content": "Which party should I vote for in the next election?"}]
    response = await rails.generate_async(messages=messages)
    print(response['content'])

    The content safety rail blocks the request.

    I'm sorry, I can't respond to that.
  4. Verify the guardrails with an on-topic question.

    messages = [{"role": "user", "content": "I'd like to cancel my subscription. Can I do this by phone or on the website?"}]
    response = await rails.generate_async(messages=messages)
    print(response['content'])

    The model responds with advice on how to cancel a subscription by phone or website.

Deploy Nemotron 3.5 Content Safety NIM Locally

This section shows how to run the Nemotron 3.5 Content Safety model locally while still using the build.nvidia.com hosted main model. The prerequisites are:

  • The NeMo Guardrails library installed.
  • A personal NVIDIA NGC API key with NVIDIA NGC Catalog and NVIDIA Public API Endpoints services access. For more information, refer to NGC API Keys in the NVIDIA GPU cloud documentation.
  • Docker installed.
  • NVIDIA Container Toolkit installed.
  • GPUs meeting the memory requirements for the Nemotron 3.5 Content Safety NIM.

To run the Nemotron 3.5 Content Safety NIM in a Docker container, follow these steps:

  1. Update the config.yml file to point to a local NIM deployment. Add a base_url field under parameters:

    models:
    - type: main
    engine: nim
    model: nvidia/nemotron-3.5-lightning-30b-a3b
    - type: content_safety
    engine: nim
    model: nvidia/nemotron-3.5-content-safety
    parameters:
    base_url: "http://localhost:8123/v1"
    chat_template_kwargs:
    enable_thinking: false
    request_categories: "/categories"
    custom_policy: |
    You are to act as a customer service agent, providing users with factual information in accordance with the knowledge base. Your role is to ensure that you respond only to relevant queries and adhere to the following guidelines:
    - Do not answer questions related to personal opinions or advice on a user's order or future recommendations.
    - Do not provide any information on non-company products or services.
    - Do not answer enquiries unrelated to the company policies.
    - Do not answer questions asking for personal details about the agent or its creators.
    - Do not answer questions about sensitive topics related to politics, religion, or other sensitive subjects.
    - If a user asks about topics irrelevant to the company's customer service, politely redirect the conversation or end the interaction.
    - Allow user comments that are related to small talk and chit-chat.
    rails:
    input:
    flows:
    - content safety check input $model=content_safety
  2. Start the Nemotron 3.5 Content Safety NIM Docker container.

    1. Export your personal NGC API key to an environment variable.

      $ export NGC_API_KEY="..."

      Log in to the NGC registry.

      $ docker login nvcr.io --username '$oauthtoken' --password-stdin <<< $NGC_API_KEY
    2. Download the container.

      $ docker pull nvcr.io/nim/nvidia/nemotron-3.5-content-safety:latest
    3. Create a model cache directory on the host machine.

      $ export LOCAL_NIM_CACHE=~/.cache/nim
      $ mkdir -p "${LOCAL_NIM_CACHE}"
      $ chmod 700 "${LOCAL_NIM_CACHE}"
    4. Run the container with the cache directory mounted.

      The -p argument maps the Docker container port 8000 to 8123 to avoid conflicts with other servers running locally.

      $ docker run -d \
      --name nemotron-content-safety \
      --gpus=all --runtime=nvidia \
      --shm-size=64GB \
      -e NGC_API_KEY \
      -u $(id -u) \
      -v "${LOCAL_NIM_CACHE}:/opt/nim/.cache/" \
      -p 8123:8000 \
      nvcr.io/nim/nvidia/nemotron-3.5-content-safety:latest

      The container requires several minutes to start and download the model from NGC. Monitor progress with docker logs nemotron-content-safety.

    5. Confirm the service is ready.

      $ curl -X GET http://localhost:8123/v1/models | jq '.data[].id'

      This returns the following response.

      "nvidia/nemotron-3.5-content-safety"
  3. Follow the steps in Run the Guardrails Chat Application and Import the NeMo Guardrails Library in Python to run Guardrails with the local model.

Next Steps