HuggingFace Classifier Integration
Content moderation using HuggingFace text classification models on input, output, and retrieval flows.
Overview
Fast, prompt-free alternative to LLM-based self-check rails. Supports four inference backends:
Setup
For the local backend, install the required dependencies:
The model is downloaded on first use from HuggingFace Hub. For air-gapped environments, set HF_HUB_OFFLINE=1 and point model to a local path.
Avoid tokenizers 0.23.1 and 0.23.2: their overflow-token regression can omit input
from both text-classification and token-classification windows. The upstream fix is
available in tokenizers 0.23.3.
With either affected version, this rail raises an error before automatic text-classification
windowing, or during pipeline initialization when token classification has stride configured
(including 0). The error identifies the incompatible versions and explains how to upgrade.
Short text-classification inputs and explicitly truncated inputs retain their native behavior.
Remote backends do not check the locally installed tokenizer version.
From a source checkout, you can apply the same exclusion with the rail-local constraints file:
The constraints file only restricts versions during this explicit installation. It does
not install packages by itself or change the project’s default or all dependencies.
For remote backends, a running inference server is required. No additional Python dependencies are needed.
Colang 2.x requires an explicit import in your Colang file (e.g., config.co):
Colang 1.0 auto-discovers library flows.
Usage
Configuration Structure
Add the classifier configuration to your config.yml:
The $classifier parameter must match the name under rails.config.hf_classifier.
Configuration Options
Common fields (all engines)
Blocked Labels
Values must match the label strings returned by the model or server. For local and vLLM backends with text-classification, labels come from the model’s id2label mapping (e.g., "toxic", "LABEL_1"). For token-classification with aggregation_strategy, labels are entity groups with the B-/I- prefix stripped (e.g., "PER", "LOC"). For FMS, labels come from the detection_type field in the server response. For KServe, labels are stringified class indices ("0", "1").
To discover labels, inspect id2label from the model config:
For remote servers, send a test request and inspect the response.
Local engine fields
Long Local Inputs
For local text-classification, inputs that exceed the inferred model capacity are
processed automatically in overlapping token windows when truncation is disabled or
unspecified. Any window with a blocked label at or above threshold triggers the rail;
scores are not averaged. Short inputs keep the original Hugging Face behavior.
Configured truncation is preserved: text beyond the truncation limit is discarded and
is not checked by this rail. Hugging Face also enables truncation implicitly when
parameters.max_length is supplied without parameters.truncation and
parameters.padding is omitted or set to false. Other padding settings do not
implicitly enable truncation; oversized inputs still use automatic windows unless
truncation is explicitly enabled.
Local token-classification uses the native Hugging Face pipeline after the version check. It normally
truncates to the tokenizer limit by default, so text beyond that limit is not checked.
Existing parameters.stride and aggregation_strategy settings continue to control
Hugging Face’s own overlapping windows and entity aggregation.
To evaluate the full input with token classification, configure overlapping windows:
This requires a fast tokenizer and an aggregation strategy other than none. Set
stride smaller than the tokenizer’s window capacity after special tokens; 0 also
processes the full input, with no overlap. Token classification does not accept
parameters.truncation; use stride to enable full-input processing.
Automatic text-classification windows require a fast tokenizer. The window limit comes
from the model’s position capacity (accounting for reserved positions), falling back to
the tokenizer’s limit. If neither declares a finite limit, native processing is retained;
set parameters.max_length with parameters.truncation: false to provide a window limit.
Overlap defaults to the smaller of 32 tokens and one quarter of the content capacity;
parameters.stride overrides it, including 0 for no overlap. Overlap must be smaller
than the window’s content capacity. Windows cannot preserve context longer than a window.
Default text-classification inference scores one window at a time. Token storage grows with input length,
and oversized inputs require a second tokenization pass. Explicit parameters.batch_size
and parameters.num_workers settings retain Hugging Face’s batching behavior and memory
tradeoffs for multiple windows. A single-window scalar input runs directly and needs no
padding token, even with a larger batch size.
Before inference, automatic windowing verifies that the windows cover the complete input token sequence. Missing or changed tokens raise an error instead of accepting a partial classification. This check does not change the native token-classification pipeline.
Automatic text-classification windowing permits at most parameters.max_windows windows
per input (default 128). It rejects inputs exceeding that budget before overflow
tokenization or inference; it does not silently truncate them. Increase the positive
integer budget to allow longer inputs. The cap bounds model forward passes, but the
initial tokenization still reads the full input and allocates storage proportional to
its length. Apply a request-size limit before invoking the rail when you also need to
bound tokenization work and memory.
Calls sharing the cached text pipeline are serialized to avoid tokenizer races, including short inputs. The window cap limits inference work per call, but does not guarantee latency; choose a budget appropriate to your model, hardware, and serving requirements. Unsupported tokenizers, invalid settings and inference failures still raise errors. Remote backends retain their server’s input-length behavior.
Remote engine fields (vllm, kserve, fms)
Input Rails
Prompt injection detection using KServe:
Output Rails
HAP detection using FMS:
Retrieval Rails
The retrieval rail classifies the combined retrieved text as a single input. If any blocked label is detected above threshold, all retrieved chunks are cleared.
Complete Example
HAP (FMS), prompt injection (KServe), and language classification (vLLM) with streaming:
Return Value
Actions return RailOutcome.allow() or RailOutcome.block() for input and output checks.
A triggered retrieval check returns a transform outcome that clears all retrieved chunks.
Triggered labels and scores are logged at INFO level:
mTLS and Custom CA
HF Classifier Rail Behavior
When blocked, input and output rails respond with "I'm sorry, I can't respond to that." and abort. If enable_rails_exceptions is set, an InputRailException or OutputRailException is raised instead. Retrieval rails clear all retrieved chunks if any blocked label is detected. With streaming enabled, the output rail checks the accumulated response after streaming completes.
Engine Support
hf classifier check retrieval clears the retrieved chunks when a blocked label is detected, which rewrites relevant_chunks.
IORails has no retrieval pipeline and nowhere to apply that rewrite, so the flow runs on LLMRails only.
This rail declares transformers as an optional dependency for its in-process backend.
IORails enforces this dependency only when the configuration selects the local backend.
With a vLLM, KServe, or FMS endpoint, the rail compiles without transformers installed.
LLMRails imports it lazily and fails on the first request instead.
For the full per-rail matrix, refer to Rail Engine Support.