> This page is for version v0.25.0.
> For other versions, use one of these documentation indexes:
> - Latest (default): https://docs.nvidia.com/nemo/guardrails/latest/llms.txt
> - v0.25.0: https://docs.nvidia.com/nemo/guardrails/v0.25.0/llms.txt
> - v0.24.1: https://docs.nvidia.com/nemo/guardrails/v0.24.1/llms.txt
> - v0.24.0: https://docs.nvidia.com/nemo/guardrails/v0.24.0/llms.txt
> - v0.23.0: https://docs.nvidia.com/nemo/guardrails/v0.23.0/llms.txt
> - v0.22.0: https://docs.nvidia.com/nemo/guardrails/v0.22.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/guardrails/llms.txt.

# HuggingFace Classifier Integration

Content moderation using HuggingFace text classification models on input, output, and retrieval flows.

## Overview

Fast, prompt-free alternative to LLM-based self-check rails. Supports four inference backends:

| Backend    | Engine   | Endpoint                               | Use Case                              |
| ---------- | -------- | -------------------------------------- | ------------------------------------- |
| **Local**  | `local`  | N/A (in-process)                       | HuggingFace Transformers pipeline     |
| **vLLM**   | `vllm`   | `{base_url}/classify`                  | vLLM classify endpoint                |
| **KServe** | `kserve` | `{base_url}/v1/models/{model}:predict` | KServe v1 predict endpoint            |
| **FMS**    | `fms`    | `{base_url}/api/v1/text/contents`      | IBM FMS guardrails-detectors endpoint |

## Setup

For the **local** backend, install the required dependencies:

```bash
pip install transformers torch "tokenizers!=0.23.1,!=0.23.2"
```

The model is downloaded on first use from HuggingFace Hub. For air-gapped environments, set `HF_HUB_OFFLINE=1` and point `model` to a local path.

Avoid tokenizers 0.23.1 and 0.23.2: their overflow-token regression can omit input
from both text-classification and token-classification windows. The upstream fix is
available in [tokenizers 0.23.3](https://github.com/huggingface/tokenizers/issues/2091).
With either affected version, this rail raises an error before automatic text-classification
windowing, or during pipeline initialization when token classification has `stride` configured
(including `0`). The error identifies the incompatible versions and explains how to upgrade.
Short text-classification inputs and explicitly truncated inputs retain their native behavior.
Remote backends do not check the locally installed tokenizer version.

From a source checkout, you can apply the same exclusion with the rail-local constraints file:

```bash
uv pip install transformers torch -c nemoguardrails/library/hf_classifier/constraints.txt
```

The constraints file only restricts versions during this explicit installation. It does
not install packages by itself or change the project's default or `all` dependencies.

For **remote** backends, a running inference server is required. No additional Python dependencies are needed.

Colang 2.x requires an explicit import in your Colang file (e.g., `config.co`):

```text
import nemoguardrails.library.hf_classifier
```

Colang 1.0 auto-discovers library flows.

## Usage

### Configuration Structure

Add the classifier configuration to your `config.yml`:

```yaml
rails:
  config:
    hf_classifier:
      named_entity_recognition:
        engine: local
        model: dslim/distilbert-NER
        task: token-classification
        threshold: 0.7
        blocked_labels:
          - "PER"
          - "LOC"
          - "ORG"
        parameters:
          aggregation_strategy: simple
  input:
    flows:
      - hf classifier check input $classifier=named_entity_recognition
  output:
    flows:
      - hf classifier check output $classifier=named_entity_recognition
```

The `$classifier` parameter must match the name under `rails.config.hf_classifier`.

### Configuration Options

**Common fields (all engines)**

| Option           | Type   | Default    | Description                                                                          |
| ---------------- | ------ | ---------- | ------------------------------------------------------------------------------------ |
| `engine`         | string | *required* | `local`, `vllm`, `kserve`, or `fms`.                                                 |
| `model`          | string | *required* | HuggingFace model ID, local path, or server-side model name.                         |
| `threshold`      | float  | `0.5`      | Minimum score to trigger blocking (0.0-1.0).                                         |
| `blocked_labels` | list   | `[]`       | Labels that trigger blocking above threshold. See [Blocked Labels](#blocked-labels). |

#### Blocked Labels

Values must match the label strings returned by the model or server. For **local** and **vLLM** backends with `text-classification`, labels come from the model's `id2label` mapping (e.g., `"toxic"`, `"LABEL_1"`). For `token-classification` with `aggregation_strategy`, labels are entity groups with the B-/I- prefix stripped (e.g., `"PER"`, `"LOC"`). For **FMS**, labels come from the `detection_type` field in the server response. For **KServe**, labels are stringified class indices (`"0"`, `"1"`).

To discover labels, inspect `id2label` from the model config:

```python
from transformers import AutoConfig
config = AutoConfig.from_pretrained("dslim/distilbert-NER")
print(config.id2label)
# {0: 'O', 1: 'B-PER', 2: 'I-PER', 3: 'B-ORG', 4: 'I-ORG', 5: 'B-LOC', 6: 'I-LOC', 7: 'B-MISC', 8: 'I-MISC'}
# With aggregation_strategy: simple, use "PER", "ORG", "LOC", "MISC" (prefix stripped)
```

For remote servers, send a test request and inspect the response.

**Local engine fields**

| Option                   | Type    | Default               | Description                                                                                                                                                                                             |
| ------------------------ | ------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `task`                   | string  | `text-classification` | Pipeline task type. Use `token-classification` for NER models.                                                                                                                                          |
| `parameters`             | dict    | `{}`                  | Kwargs forwarded to `transformers.pipeline()`.                                                                                                                                                          |
| `parameters.max_windows` | integer | `128`                 | Positive maximum number of automatic text-classification windows per input. Inputs exceeding the budget raise an error before inference. Does not apply to token classification or explicit truncation. |

#### Long Local Inputs

For local `text-classification`, inputs that exceed the inferred model capacity are
processed automatically in overlapping token windows when truncation is disabled or
unspecified. Any window with a blocked label at or above `threshold` triggers the rail;
scores are not averaged. Short inputs keep the original Hugging Face behavior.

Configured truncation is preserved: text beyond the truncation limit is discarded and
is **not** checked by this rail. Hugging Face also enables truncation implicitly when
`parameters.max_length` is supplied without `parameters.truncation` and
`parameters.padding` is omitted or set to `false`. Other padding settings do not
implicitly enable truncation; oversized inputs still use automatic windows unless
truncation is explicitly enabled.

Local `token-classification` uses the native Hugging Face pipeline after the version check. It normally
truncates to the tokenizer limit by default, so text beyond that limit is not checked.
Existing `parameters.stride` and `aggregation_strategy` settings continue to control
Hugging Face's own overlapping windows and entity aggregation.

To evaluate the full input with token classification, configure overlapping windows:

```yaml
task: token-classification
parameters:
  aggregation_strategy: simple
  stride: 32
```

This requires a fast tokenizer and an aggregation strategy other than `none`. Set
`stride` smaller than the tokenizer's window capacity after special tokens; `0` also
processes the full input, with no overlap. Token classification does not accept
`parameters.truncation`; use `stride` to enable full-input processing.

Automatic text-classification windows require a fast tokenizer. The window limit comes
from the model's position capacity (accounting for reserved positions), falling back to
the tokenizer's limit. If neither declares a finite limit, native processing is retained;
set `parameters.max_length` with `parameters.truncation: false` to provide a window limit.
Overlap defaults to the smaller of 32 tokens and one quarter of the content capacity;
`parameters.stride` overrides it, including `0` for no overlap. Overlap must be smaller
than the window's content capacity. Windows cannot preserve context longer than a window.

Default text-classification inference scores one window at a time. Token storage grows with input length,
and oversized inputs require a second tokenization pass. Explicit `parameters.batch_size`
and `parameters.num_workers` settings retain Hugging Face's batching behavior and memory
tradeoffs for multiple windows. A single-window scalar input runs directly and needs no
padding token, even with a larger batch size.

Before inference, automatic windowing verifies that the windows cover the complete input
token sequence. Missing or changed tokens raise an error instead of accepting a partial
classification. This check does not change the native token-classification pipeline.

Automatic text-classification windowing permits at most `parameters.max_windows` windows
per input (default `128`). It rejects inputs exceeding that budget before overflow
tokenization or inference; it does not silently truncate them. Increase the positive
integer budget to allow longer inputs. The cap bounds model forward passes, but the
initial tokenization still reads the full input and allocates storage proportional to
its length. Apply a request-size limit before invoking the rail when you also need to
bound tokenization work and memory.

Calls sharing the cached text pipeline are serialized to avoid tokenizer races, including
short inputs. The window cap limits inference work per call, but does not guarantee latency;
choose a budget appropriate to your model, hardware, and serving requirements.
Unsupported tokenizers, invalid settings and inference failures still raise errors.
Remote backends retain their server's input-length behavior.

**Remote engine fields (vllm, kserve, fms)**

| Option                   | Type   | Default    | Description                                       |
| ------------------------ | ------ | ---------- | ------------------------------------------------- |
| `base_url`               | string | *required* | Inference server URL.                             |
| `api_key_env_var`        | string | `null`     | Environment variable name holding the API key.    |
| `parameters.timeout`     | float  | `30.0`     | Request timeout in seconds.                       |
| `parameters.verify_ssl`  | bool   | `true`     | Set `false` to skip TLS verification.             |
| `parameters.ca_cert`     | string | `null`     | CA bundle path for custom CAs.                    |
| `parameters.client_cert` | string | `null`     | Client certificate path for mTLS.                 |
| `parameters.client_key`  | string | `null`     | Client key path for mTLS. Requires `client_cert`. |

### Input Rails

Prompt injection detection using KServe:

```yaml
rails:
  config:
    hf_classifier:
      prompt_injection:
        engine: kserve
        model: prompt-injection-detector
        base_url: "https://prompt-injection-detector-route.apps.example.com"
        api_key_env_var: OCP_TOKEN
        threshold: 0.5
        blocked_labels:
          - "1"
        parameters:
          verify_ssl: false
  input:
    flows:
      - hf classifier check input $classifier=prompt_injection
```

### Output Rails

HAP detection using FMS:

```yaml
rails:
  config:
    hf_classifier:
      hap:
        engine: fms
        model: hap-detector
        base_url: "https://detector-hap-route.apps.example.com"
        api_key_env_var: OCP_TOKEN
        threshold: 0.7
        blocked_labels:
          - "LABEL_1"
        parameters:
          verify_ssl: false
  output:
    flows:
      - hf classifier check output $classifier=hap
```

### Retrieval Rails

The retrieval rail classifies the combined retrieved text as a single input. If any blocked label is detected above threshold, **all** retrieved chunks are cleared.

```yaml
rails:
  config:
    hf_classifier:
      named_entity_recognition:
        engine: local
        model: dslim/distilbert-NER
        task: token-classification
        threshold: 0.7
        blocked_labels:
          - "PER"
          - "LOC"
          - "ORG"
        parameters:
          aggregation_strategy: simple
  retrieval:
    flows:
      - hf classifier check retrieval $classifier=named_entity_recognition
```

## Complete Example

HAP (FMS), prompt injection (KServe), and language classification (vLLM) with streaming:

```yaml
models:
  - type: main
    engine: openai
    model: my-model
    parameters:
      base_url: "https://llm-server.apps.example.com/v1"

rails:
  config:
    hf_classifier:
      hap:
        engine: fms
        model: hap-detector
        base_url: "https://detector-hap-route.apps.example.com"
        api_key_env_var: OCP_TOKEN
        threshold: 0.7
        blocked_labels:
          - "LABEL_1"
        parameters:
          verify_ssl: false

      prompt_injection:
        engine: kserve
        model: prompt-injection-detector
        base_url: "https://prompt-injection-detector-route.apps.example.com"
        api_key_env_var: OCP_TOKEN
        threshold: 0.5
        blocked_labels:
          - "1"
        parameters:
          verify_ssl: false

      lang:
        engine: vllm
        model: language-classifier
        base_url: "https://language-classifier-route.apps.example.com"
        api_key_env_var: OCP_TOKEN
        threshold: 0.5
        blocked_labels:
          - "fr"
          - "de"
          - "es"
        parameters:
          verify_ssl: false

  input:
    flows:
      - hf classifier check input $classifier=prompt_injection
      - hf classifier check input $classifier=hap
      - hf classifier check input $classifier=lang
  output:
    flows:
      - hf classifier check output $classifier=hap
    streaming:
      enabled: true
      stream_first: false
```

## Return Value

Actions return `RailOutcome.allow()` or `RailOutcome.block()` for input and output checks.
A triggered retrieval check returns a transform outcome that clears all retrieved chunks.
Triggered labels and scores are logged at `INFO` level:

```text
HF classifier 'hap': blocked (detections: [('LABEL_1', 0.92)])
```

## mTLS and Custom CA

```yaml
rails:
  config:
    hf_classifier:
      toxicity:
        engine: kserve
        model: toxic-bert
        base_url: "https://classifier.internal:443"
        threshold: 0.7
        blocked_labels:
          - toxic
        parameters:
          ca_cert: /etc/ssl/custom-ca.pem
          client_cert: /etc/ssl/client.pem
          client_key: /etc/ssl/client.key
```

## HF Classifier Rail Behavior

When blocked, **input and output rails** respond with `"I'm sorry, I can't respond to that."` and abort. If `enable_rails_exceptions` is set, an `InputRailException` or `OutputRailException` is raised instead. **Retrieval rails** clear all retrieved chunks if any blocked label is detected. With streaming enabled, the output rail checks the accumulated response after streaming completes.

## Engine Support

| Flow                            | LLMRails | IORails |
| ------------------------------- | :------: | :-----: |
| `hf classifier check input`     |     ✓    |    ✓    |
| `hf classifier check output`    |     ✓    |    ✓    |
| `hf classifier check retrieval` |     ✓    |    ✗    |

`hf classifier check retrieval` clears the retrieved chunks when a blocked label is detected, which rewrites `relevant_chunks`.
`IORails` has no retrieval pipeline and nowhere to apply that rewrite, so the flow runs on `LLMRails` only.

This rail declares `transformers` as an optional dependency for its in-process backend.
`IORails` enforces this dependency only when the configuration selects the local backend.
With a vLLM, KServe, or FMS endpoint, the rail compiles without `transformers` installed.
`LLMRails` imports it lazily and fails on the first request instead.

For the full per-rail matrix, refer to [Rail Engine Support](/nemo/guardrails/v0.25.0/reference/rail-engine-support).