Guardrails Configuration Store#

A guardrail configuration specifies information such as the models, model endpoints, and the checks to apply to input and output. In each guardrails inference request, you can specify the configuration by providing the configuration ID.

The NeMo Guardrails microservice supports two storage backends for the guardrail configurations that you create.

  1. PostgreSQL Database: This is the default option.

  2. External PostgreSQL Database: This is an option for an existing database infrastructure and is recommended for production deployments.

Each option provides different benefits and trade-offs in terms of scalability, reliability, and ease of management.

Set Up PostgreSQL for Storing Guardrails Configurations#

While deploying the service on Kubernetes, you can either use the default PostgreSQL or configure an external production-grade PostgreSQL. To learn more about setting up PostgreSQL for the NeMo microservices in general, refer to PostgreSQL.

Option 1: Default PostgreSQL#

The default PostgreSQL is automatically provisioned and managed by the Helm chart. The NeMo Microservices Helm Chart uses the Bitnami PostgreSQL chart to deploy a PostgreSQL database.

guardrails:
  postgresql:
    enabled: true

Preload Guardrail Configurations From the Values File#

After starting the microservice, you can manage configurations through the REST API. However, if you want to preload configurations into the PostgreSQL database, you can do so by also updating the values.yaml file. When the microservice starts, it saves the configurations you provide in the values.yaml file to the database.

Option 1: NFS Storage#

Use NFS storage when you need file-based persistence or cannot use PostgreSQL. This option allows you to load configurations from files in a shared filesystem. The following is the default configuration in the values.yaml file:

guardrails:
  configStore:
    nfs:
      enabled: false
      path: "/path/to/nfs/share"
      server: "nfs-server.example.com"
      mountPath: "/config-store"
      storageClass: "standard"
  env:
    CONFIG_STORE_PATH: "/app/services/guardrails/config-store"

Make sure that the configStore.nfs.path key references the root of the configuration store directory, which is the directory that contains subfolders for each individual guardrail configuration.

The CONFIG_STORE_PATH environment variable points to the directory in the microservice container that contains your configurations.

Option 2: In-Line YAML Object#

You can define one or more guardrail configurations directly in your values.yaml file as a YAML object. The key in the files object must be formatted as <namespace>/<guardrail-config-name>/config.yaml. If you omit the namespace, the configuration uses default as the namespace by default.

Note

The namespace for guardrail configurations here is the NeMo microservices platform namespace and not associated with Kubernetes namespace.

The following example shows a complete guardrail configuration with content safety and topic control models:

guardrails:
  configStore:
    files:
      "nemoguard/config.yaml":
        data:
          models:
          - type: content_safety
            engine: nim
            model: llama-3.1-nemoguard-8b-content-safety
            parameters:
              base_url: http://content-safety-nim-llm:8000/v1
          - type: topic_control
            engine: nim
            model: llama-3.1-nemoguard-8b-topic-control
            parameters:
              base_url: http://topic-control-nim-llm:8000/v1
          rails:
            input:
              parallel: false
              flows:
              - content safety check input $model=content_safety
              - content safety check input $model=content_safety
              - topic safety check input $model=topic_control
            output:
              parallel: false
              flows:
              - content safety check output $model=content_safety
          prompts:
          - task: content_safety_check_input $model=content_safety
            content: |
              Task: Check if there is unsafe content in the user or agent messages in the below conversation in accordance with our safety policy with the below categories. Provide a 'safe' or 'unsafe' rating for both the user message and the agent response. If there is unsafe content, please also provide a list of violated categories according to our safety policy below.

              <BEGIN UNSAFE CONTENT CATEGORIES>
              S1: Violence.
              S2: Sexual.
              S3: Criminal Planning/Confessions.
              S4: Guns and Illegal Weapons.
              S5: Controlled/Regulated Substances.
              S6: Suicide and Self Harm.
              S7: Sexual (minor).
              S8: Hate/Identity Hate.
              S9: PII/Privacy.
              S10: Harassment.
              S11: Threat.
              S12: Profanity.
              S13: Needs Caution.
              S14: Other.
              S15: Manipulation.
              S16: Fraud/Deception.
              S17: Malware.
              S18: High Risk Gov Decision Making.
              S19: Political/Misinformation/Conspiracy.
              S20: Copyright/Trademark/Plagiarism.
              S21: Unauthorized Advice.
              S22: Illegal Activity.
              S23: Immoral/Unethical.
              <END UNSAFE CONTENT CATEGORIES>

              <BEGIN CONVERSATION>

              user: {{ user_input }}

              <END CONVERSATION>

              Provide your safety assessment for the above conversation in the below output JSON format:
              {"User Safety": safe or unsafe rating for the user message, "Response Safety": safe or unsafe rating for the agent response. Omit if no agent response present. "Safety Categories": a comma-separated list of applicable safety categories from the provided taxonomy. Omit if all safe.}

              Do not include anything other than the output JSON in your response.
              Output JSON:
            output_parser: nemoguard_parse_prompt_safety
            max_tokens: 50

          - task: content_safety_check_output $model=content_safety
            content: |
              Task: Check if there is unsafe content in the user or agent messages in the below conversation in accordance with our safety policy with the below categories. Provide a 'safe' or 'unsafe' rating for both the user message and the agent response. If there is unsafe content, please also provide a list of violated categories according to our safety policy below.

              <BEGIN UNSAFE CONTENT CATEGORIES>
              S1: Violence.
              S2: Sexual.
              S3: Criminal Planning/Confessions.
              S4: Guns and Illegal Weapons.
              S5: Controlled/Regulated Substances.
              S6: Suicide and Self Harm.
              S7: Sexual (minor).
              S8: Hate/Identity Hate.
              S9: PII/Privacy.
              S10: Harassment.
              S11: Threat.
              S12: Profanity.
              S13: Needs Caution.
              S14: Other.
              S15: Manipulation.
              S16: Fraud/Deception.
              S17: Malware.
              S18: High Risk Gov Decision Making.
              S19: Political/Misinformation/Conspiracy.
              S20: Copyright/Trademark/Plagiarism.
              S21: Unauthorized Advice.
              S22: Illegal Activity.
              S23: Immoral/Unethical.
              <END UNSAFE CONTENT CATEGORIES>

              <BEGIN CONVERSATION>

              user: {{ user_input }}

              response: agent: {{ bot_response }}

              <END CONVERSATION>

              Provide your safety assessment for the above conversation in the below output JSON format:
              {"User Safety": safe or unsafe rating for the user message, "Response Safety": safe or unsafe rating for the agent response. Omit if no agent response present. "Safety Categories": a comma-separated list of applicable safety categories from the provided taxonomy. Omit if all safe.}

              Do not include anything other than the output JSON in your response.
              Output JSON:
            output_parser: nemoguard_parse_response_safety
            max_tokens: 50

          - task: topic_safety_check_input $model=topic_control
            content: |
              You are to act as a customer service agent, providing users with factual information in accordance to the knowledge base. Your role is to ensure that you respond only to relevant queries and adhere to the following guidelines

              Guidelines for the user messages:
              - Do not answer questions related to personal opinions or advice on user's order, future recommendations
              - Do not provide any information on non-company products or services.
              - Do not answer enquiries unrelated to the company policies.
              - Do not answer questions asking for personal details about the agent or its creators.
              - Do not answer questions about sensitive topics related to politics, religion, or other sensitive subjects.
              - If a user asks topics irrelevant to the company's customer service relations, politely redirect the conversation or end the interaction.
              - Your responses should be professional, accurate, and compliant with customer relations guidelines, focusing solely on providing transparent, up-to-date information about the company that is already publicly available.
              - allow user comments that are related to small talk and chit-chat.

          - task: topic_safety_check_output $model=topic_control
            content: |
              You are to act as a customer service agent, providing users with factual information in accordance to the knowledge base. Your role is to ensure that you respond only to relevant queries and adhere to the following guidelines

              Guidelines for the user messages:
              - Do not answer questions related to personal opinions or advice on user's order, future recommendations
              - Do not provide any information on non-company products or services.
              - Do not answer enquiries unrelated to the company policies.
              - Do not answer questions asking for personal details about the agent or its creators.
              - Do not answer questions about sensitive topics related to politics, religion, or other sensitive subjects.
              - If a user asks topics irrelevant to the company's customer service relations, politely redirect the conversation or end the interaction.
              - Your responses should be professional, accurate, and compliant with customer relations guidelines, focusing solely on providing transparent, up-to-date information about the company that is already publicly available.
              - allow user comments that are related to small talk and chit-chat.

The configuration ID of the guardrail configuration in the example above is default/nemoguard because a namespace is not specified in the key "nemoguard/config.yaml".

Environment Variable Configuration#

Configure the configuration store path and related settings through environment variables. The following are the default values in the values.yaml file:

guardrails:
  env:
    CONFIG_STORE_PATH: "/app/services/guardrails/config-store"
    DEMO: "True"
    DEFAULT_CONFIG_ID: self-check

Demo Configurations#

The NeMo Guardrails microservice includes a default guardrails configuration store pre-populated with three configurations.

When you start the microservice with the DEMO environment variable set to True, which is the default value, the guardrails configuration store is initialized with the following three configurations.

  • default: This is an empty guardrail configuration, useful for testing the connectivity to the LLM.

  • self-check: This is a demo guardrail configuration using a simple self-check input rail. For more details, check out the Self Check Input section in the NeMo Guardrails toolkit documentation.

  • abc: This configures the ABC Bot in the NeMo Guardrails toolkit. The bot answers questions about the ABC company employee handbook.

The microservice adds a separate directory for each configuration to the locally accessible directory for the configuration store. For the demo configurations, the layout is as follows:

/config-store
├── abc
│   ├── rails
│   │   └── disallowed.co
│   ├── kb
│   │   └── employee-handbook.md
│   ├── prompts.yml
│   └── config.yml
├── default
│   └── config.yml
└── self-check
    └── config.yml

With the default DEFAULT_CONFIG_ID set to self-check, the NeMo Guardrails microservice uses the self-check configuration.