HuggingFace Classifier Integration | NVIDIA NeMo Guardrails Library Developer Guide

Content moderation using HuggingFace text classification models on input, output, and retrieval flows.

Overview

Fast, prompt-free alternative to LLM-based self-check rails. Supports four inference backends:

Backend	Engine	Endpoint	Use Case
Local	`local`	N/A (in-process)	HuggingFace Transformers pipeline
vLLM	`vllm`	`{base_url}/classify`	vLLM classify endpoint
KServe	`kserve`	`{base_url}/v1/models/{model}:predict`	KServe v1 predict endpoint
FMS	`fms`	`{base_url}/api/v1/text/contents`	IBM FMS guardrails-detectors endpoint

Setup

For the local backend, install the required dependencies:

$ pip install transformers torch

The model is downloaded on first use from HuggingFace Hub. For air-gapped environments, set HF_HUB_OFFLINE=1 and point model to a local path.

For remote backends, a running inference server is required. No additional Python dependencies are needed.

Colang 2.x requires an explicit import in your Colang file (e.g., config.co):

import nemoguardrails.library.hf_classifier

Colang 1.0 auto-discovers library flows.

Usage

Configuration Structure

Add the classifier configuration to your config.yml:

1 rails:
2   config:
3     hf_classifier:
4       named_entity_recognition:
5         engine: local
6         model: dslim/distilbert-NER
7         task: token-classification
8         threshold: 0.7
9         blocked_labels:
10           - "PER"
11           - "LOC"
12           - "ORG"
13         parameters:
14           aggregation_strategy: simple
15   input:
16     flows:
17       - hf classifier check input $classifier=named_entity_recognition
18   output:
19     flows:
20       - hf classifier check output $classifier=named_entity_recognition

The $classifier parameter must match the name under rails.config.hf_classifier.

Configuration Options

Common fields (all engines)

Option	Type	Default	Description
`engine`	string	required	`local`, `vllm`, `kserve`, or `fms`.
`model`	string	required	HuggingFace model ID, local path, or server-side model name.
`threshold`	float	`0.5`	Minimum score to trigger blocking (0.0-1.0).
`blocked_labels`	list	`[]`	Labels that trigger blocking above threshold. See Blocked Labels.

Blocked Labels

Values must match the label strings returned by the model or server. For local and vLLM backends with text-classification, labels come from the model’s id2label mapping (e.g., "toxic", "LABEL_1"). For token-classification with aggregation_strategy, labels are entity groups with the B-/I- prefix stripped (e.g., "PER", "LOC"). For FMS, labels come from the detection_type field in the server response. For KServe, labels are stringified class indices ("0", "1").

To discover labels, inspect id2label from the model config:

1 from transformers import AutoConfig
2 config = AutoConfig.from_pretrained("dslim/distilbert-NER")
3 print(config.id2label)
4 # {0: 'O', 1: 'B-PER', 2: 'I-PER', 3: 'B-ORG', 4: 'I-ORG', 5: 'B-LOC', 6: 'I-LOC', 7: 'B-MISC', 8: 'I-MISC'}
5 # With aggregation_strategy: simple, use "PER", "ORG", "LOC", "MISC" (prefix stripped)

For remote servers, send a test request and inspect the response.

Local engine fields

Option	Type	Default	Description
`task`	string	`text-classification`	Pipeline task type. Use `token-classification` for NER models.
`parameters`	dict	`{}`	Kwargs forwarded to `transformers.pipeline()`.

Remote engine fields (vllm, kserve, fms)

Option	Type	Default	Description
`base_url`	string	required	Inference server URL.
`api_key_env_var`	string	`null`	Environment variable name holding the API key.
`parameters.timeout`	float	`30.0`	Request timeout in seconds.
`parameters.verify_ssl`	bool	`true`	Set `false` to skip TLS verification.
`parameters.ca_cert`	string	`null`	CA bundle path for custom CAs.
`parameters.client_cert`	string	`null`	Client certificate path for mTLS.
`parameters.client_key`	string	`null`	Client key path for mTLS. Requires `client_cert`.

Input Rails

Prompt injection detection using KServe:

1 rails:
2   config:
3     hf_classifier:
4       prompt_injection:
5         engine: kserve
6         model: prompt-injection-detector
7         base_url: "https://prompt-injection-detector-route.apps.example.com"
8         api_key_env_var: OCP_TOKEN
9         threshold: 0.5
10         blocked_labels:
11           - "1"
12         parameters:
13           verify_ssl: false
14   input:
15     flows:
16       - hf classifier check input $classifier=prompt_injection

Output Rails

HAP detection using FMS:

1 rails:
2   config:
3     hf_classifier:
4       hap:
5         engine: fms
6         model: hap-detector
7         base_url: "https://detector-hap-route.apps.example.com"
8         api_key_env_var: OCP_TOKEN
9         threshold: 0.7
10         blocked_labels:
11           - "LABEL_1"
12         parameters:
13           verify_ssl: false
14   output:
15     flows:
16       - hf classifier check output $classifier=hap

Retrieval Rails

The retrieval rail classifies the combined retrieved text as a single input. If any blocked label is detected above threshold, all retrieved chunks are cleared.

1 rails:
2   config:
3     hf_classifier:
4       named_entity_recognition:
5         engine: local
6         model: dslim/distilbert-NER
7         task: token-classification
8         threshold: 0.7
9         blocked_labels:
10           - "PER"
11           - "LOC"
12           - "ORG"
13         parameters:
14           aggregation_strategy: simple
15   retrieval:
16     flows:
17       - hf classifier check retrieval $classifier=named_entity_recognition

Complete Example

HAP (FMS), prompt injection (KServe), and language classification (vLLM) with streaming:

1 models:
2   - type: main
3     engine: openai
4     model: my-model
5     parameters:
6       base_url: "https://llm-server.apps.example.com/v1"
7 
8 rails:
9   config:
10     hf_classifier:
11       hap:
12         engine: fms
13         model: hap-detector
14         base_url: "https://detector-hap-route.apps.example.com"
15         api_key_env_var: OCP_TOKEN
16         threshold: 0.7
17         blocked_labels:
18           - "LABEL_1"
19         parameters:
20           verify_ssl: false
21 
22       prompt_injection:
23         engine: kserve
24         model: prompt-injection-detector
25         base_url: "https://prompt-injection-detector-route.apps.example.com"
26         api_key_env_var: OCP_TOKEN
27         threshold: 0.5
28         blocked_labels:
29           - "1"
30         parameters:
31           verify_ssl: false
32 
33       lang:
34         engine: vllm
35         model: language-classifier
36         base_url: "https://language-classifier-route.apps.example.com"
37         api_key_env_var: OCP_TOKEN
38         threshold: 0.5
39         blocked_labels:
40           - "fr"
41           - "de"
42           - "es"
43         parameters:
44           verify_ssl: false
45 
46   input:
47     flows:
48       - hf classifier check input $classifier=prompt_injection
49       - hf classifier check input $classifier=hap
50       - hf classifier check input $classifier=lang
51   output:
52     flows:
53       - hf classifier check output $classifier=hap
54     streaming:
55       enabled: true
56       stream_first: false

Return Value

Returns True if allowed, False if blocked. Triggered labels and scores are logged at INFO level:

HF classifier 'hap': blocked (detections: [('LABEL_1', 0.92)])

mTLS and Custom CA

1 rails:
2   config:
3     hf_classifier:
4       toxicity:
5         engine: kserve
6         model: toxic-bert
7         base_url: "https://classifier.internal:443"
8         threshold: 0.7
9         blocked_labels:
10           - toxic
11         parameters:
12           ca_cert: /etc/ssl/custom-ca.pem
13           client_cert: /etc/ssl/client.pem
14           client_key: /etc/ssl/client.key

HF Classifier Rail Behavior

When blocked, input and output rails respond with "I'm sorry, I can't respond to that." and abort. If enable_rails_exceptions is set, an InputRailException or OutputRailException is raised instead. Retrieval rails clear all retrieved chunks if any blocked label is detected. With streaming enabled, the output rail checks the accumulated response after streaming completes.

Engine Support

Flow	LLMRails	IORails
`hf classifier check input`	✓	✓
`hf classifier check output`	✓	✓
`hf classifier check retrieval`	✓	✗

hf classifier check retrieval clears the retrieved chunks when a blocked label is detected, which rewrites relevant_chunks. IORails has no retrieval pipeline and nowhere to apply that rewrite, so the flow runs on LLMRails only.

This rail declares transformers as an optional dependency for its in-process backend. IORails enforces this dependency only when the configuration selects the local backend. With a vLLM, KServe, or FMS endpoint, the rail compiles without transformers installed. LLMRails imports it lazily and fails on the first request instead.

For the full per-rail matrix, refer to Rail Engine Support.