> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/guardrails/llms.txt.
> For full documentation content, see https://docs.nvidia.com/nemo/guardrails/llms-full.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/guardrails/_mcp/server.

# nemoguardrails.library.hf_classifier.backends

Pluggable inference backends for HuggingFace classifier rails.

## Module Contents

### Classes

| Name                                                                                          | Description                                                              |
| --------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ |
| [`ClassificationResult`](#nemoguardrails-library-hf_classifier-backends-ClassificationResult) | -                                                                        |
| [`ClassifierBackend`](#nemoguardrails-library-hf_classifier-backends-ClassifierBackend)       | Abstract interface for HuggingFace classifier inference.                 |
| [`FMSBackend`](#nemoguardrails-library-hf_classifier-backends-FMSBackend)                     | FMS guardrails-detectors `/api/v1/text/contents` backend.                |
| [`KServeBackend`](#nemoguardrails-library-hf_classifier-backends-KServeBackend)               | KServe v1 inference predict backend.                                     |
| [`LocalBackend`](#nemoguardrails-library-hf_classifier-backends-LocalBackend)                 | Local HF Transformers pipeline backend.                                  |
| [`VLLMBackend`](#nemoguardrails-library-hf_classifier-backends-VLLMBackend)                   | vLLM `/classify` endpoint backend.                                       |
| [`_HTTPXSSLConfig`](#nemoguardrails-library-hf_classifier-backends-_HTTPXSSLConfig)           | Stores httpx-compatible SSL parameters (verify, cert).                   |
| [`_PipelineLoadError`](#nemoguardrails-library-hf_classifier-backends-_PipelineLoadError)     | Sentinel stored in \_pipelines when a pipeline fails to load at startup. |
| [`_PipelineLoading`](#nemoguardrails-library-hf_classifier-backends-_PipelineLoading)         | Sentinel stored while a pipeline is being loaded.                        |
| [`_RemoteBackend`](#nemoguardrails-library-hf_classifier-backends-_RemoteBackend)             | Base class for remote HTTP classifier backends.                          |

### Functions

| Name                                                                                                | Description                                                                       |
| --------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- |
| [`_build_headers`](#nemoguardrails-library-hf_classifier-backends-_build_headers)                   | Build HTTP request headers from classifier config.                                |
| [`_build_ssl_config`](#nemoguardrails-library-hf_classifier-backends-_build_ssl_config)             | Build httpx SSL params from config parameters (cached).                           |
| [`_check_cached`](#nemoguardrails-library-hf_classifier-backends-_check_cached)                     | Return cached pipeline, wait on loading, or raise on error. Returns None on miss. |
| [`_flatten_ints`](#nemoguardrails-library-hf_classifier-backends-_flatten_ints)                     | -                                                                                 |
| [`_get_or_create_pipeline`](#nemoguardrails-library-hf_classifier-backends-_get_or_create_pipeline) | -                                                                                 |
| [`_get_timeout`](#nemoguardrails-library-hf_classifier-backends-_get_timeout)                       | -                                                                                 |
| [`_pipeline_cache_key`](#nemoguardrails-library-hf_classifier-backends-_pipeline_cache_key)         | -                                                                                 |
| [`get_backend`](#nemoguardrails-library-hf_classifier-backends-get_backend)                         | Get or create a cached backend instance from classifier config.                   |

### Data

[`_BACKENDS`](#nemoguardrails-library-hf_classifier-backends-_BACKENDS)

[`_DEFAULT_TIMEOUT`](#nemoguardrails-library-hf_classifier-backends-_DEFAULT_TIMEOUT)

[`_FAILED_LOAD_MSG`](#nemoguardrails-library-hf_classifier-backends-_FAILED_LOAD_MSG)

[`_HTTP_ONLY_PARAMS`](#nemoguardrails-library-hf_classifier-backends-_HTTP_ONLY_PARAMS)

[`_TRANSIENT_ERRORS`](#nemoguardrails-library-hf_classifier-backends-_TRANSIENT_ERRORS)

[`_backend_instances`](#nemoguardrails-library-hf_classifier-backends-_backend_instances)

[`_pipelines`](#nemoguardrails-library-hf_classifier-backends-_pipelines)

[`_pipelines_lock`](#nemoguardrails-library-hf_classifier-backends-_pipelines_lock)

[`_ssl_cache`](#nemoguardrails-library-hf_classifier-backends-_ssl_cache)

[`_warned_env_vars`](#nemoguardrails-library-hf_classifier-backends-_warned_env_vars)

[`log`](#nemoguardrails-library-hf_classifier-backends-log)

### API

```python
class nemoguardrails.library.hf_classifier.backends.ClassificationResult
```

**Bases:** `typing.TypedDict`

```python
class nemoguardrails.library.hf_classifier.backends.ClassifierBackend()
```

Abstract

**Bases:** `ABC`

Abstract interface for HuggingFace classifier inference.

```python
nemoguardrails.library.hf_classifier.backends.ClassifierBackend.classify(
    text: str
) -> typing.List[nemoguardrails.library.hf_classifier.backends.ClassificationResult]
```

async

abstract

Classify a single text. Returns list of label/score detections.

```python
nemoguardrails.library.hf_classifier.backends.ClassifierBackend.close() -> None
```

async

Release resources (e.g. HTTP sessions). Override in subclasses.

```python
class nemoguardrails.library.hf_classifier.backends.FMSBackend(
    config: nemoguardrails.rails.llm.config.RemoteHFClassifierConfig
)
```

**Bases:** [\_RemoteBackend](#nemoguardrails-library-hf_classifier-backends-_RemoteBackend)

FMS guardrails-detectors `/api/v1/text/contents` backend.

Tested against: FMS guardrails-detectors API v1. Expects response shape::

\[\[\{"detection\_type": "...", "score": float}, ...]]

Raises `ValueError` if the response is not a list-of-lists or detection
entries lack required keys.

```python
nemoguardrails.library.hf_classifier.backends.FMSBackend.classify(
    text: str
) -> typing.List[nemoguardrails.library.hf_classifier.backends.ClassificationResult]
```

async

```python
class nemoguardrails.library.hf_classifier.backends.KServeBackend(
    config: nemoguardrails.rails.llm.config.RemoteHFClassifierConfig
)
```

**Bases:** [\_RemoteBackend](#nemoguardrails-library-hf_classifier-backends-_RemoteBackend)

KServe v1 inference predict backend.

Tested against: KServe v1 predict API (`/v1/models/{name}:predict`).
Handles three `predictions` shapes:

* `dict`: class-index → probability (`--return_probabilities`)
* `int/float`: argmax class index, score assumed 1.0
* nested `list`: token-level class indices, flattened and deduplicated

Raises `ValueError` if `predictions` key is missing or the prediction
has an unrecognised type.

```python
nemoguardrails.library.hf_classifier.backends.KServeBackend.classify(
    text: str
) -> typing.List[nemoguardrails.library.hf_classifier.backends.ClassificationResult]
```

async

```python
class nemoguardrails.library.hf_classifier.backends.LocalBackend(
    config: nemoguardrails.rails.llm.config.LocalHFClassifierConfig
)
```

**Bases:** [ClassifierBackend](#nemoguardrails-library-hf_classifier-backends-ClassifierBackend)

Local HF Transformers pipeline backend.

Tested against: `transformers >= 4.35` pipeline API for
`text-classification` and `token-classification` tasks.

```python
nemoguardrails.library.hf_classifier.backends.LocalBackend.classify(
    text: str
) -> typing.List[nemoguardrails.library.hf_classifier.backends.ClassificationResult]
```

async

```python
class nemoguardrails.library.hf_classifier.backends.VLLMBackend(
    config: nemoguardrails.rails.llm.config.RemoteHFClassifierConfig
)
```

**Bases:** [\_RemoteBackend](#nemoguardrails-library-hf_classifier-backends-_RemoteBackend)

vLLM `/classify` endpoint backend.

Tested against: vLLM v0.6.x `/classify` API. Expects response shape::

\{"data": \[\{"label": "...", "probs": \[float, ...]}]}

Raises `ValueError` if the response is missing required keys (`data`,
`label`), indicating an API change.

```python
nemoguardrails.library.hf_classifier.backends.VLLMBackend.classify(
    text: str
) -> typing.List[nemoguardrails.library.hf_classifier.backends.ClassificationResult]
```

async

```python
class nemoguardrails.library.hf_classifier.backends._HTTPXSSLConfig(
    verify: typing.Union[str, bool] = True,
    cert: typing.Optional[tuple] = None
)
```

Stores httpx-compatible SSL parameters (verify, cert).

```python
class nemoguardrails.library.hf_classifier.backends._PipelineLoadError(
    error: str
)
```

Sentinel stored in \_pipelines when a pipeline fails to load at startup.
Prevents silent retry — classify() raises immediately with the original error.

```python
class nemoguardrails.library.hf_classifier.backends._PipelineLoading()
```

Sentinel stored while a pipeline is being loaded.
Prevents duplicate concurrent loads — a second caller waits on the event.

```python
nemoguardrails.library.hf_classifier.backends._PipelineLoading.wait() -> typing.Any
```

```python
class nemoguardrails.library.hf_classifier.backends._RemoteBackend(
    config: nemoguardrails.rails.llm.config.RemoteHFClassifierConfig
)
```

**Bases:** [ClassifierBackend](#nemoguardrails-library-hf_classifier-backends-ClassifierBackend)

Base class for remote HTTP classifier backends.

Uses a fresh httpx client per request — no connection pool state to manage.
Classifier calls are infrequent and latency-tolerant (inference >> TLS
handshake), so connection reuse provides negligible benefit while introducing
stale-connection complexity.

```python
nemoguardrails.library.hf_classifier.backends._RemoteBackend._post(
    url: str,
    json: typing.Dict[str, typing.Any]
) -> httpx.Response
```

async

POST with one automatic retry on transient failures.

```python
nemoguardrails.library.hf_classifier.backends._build_headers(
    config: nemoguardrails.rails.llm.config.RemoteHFClassifierConfig
) -> typing.Dict[str, str]
```

Build HTTP request headers from classifier config.

```python
nemoguardrails.library.hf_classifier.backends._build_ssl_config(
    config: nemoguardrails.rails.llm.config.RemoteHFClassifierConfig
) -> nemoguardrails.library.hf_classifier.backends._HTTPXSSLConfig
```

Build httpx SSL params from config parameters (cached).

Reads from `config.parameters`:

* `verify_ssl` (bool, default True): set to False to skip TLS verification.
* `ca_cert` (str): path to a CA bundle file for custom/internal CAs.
* `client_cert` (str) + `client_key` (str): paths for mTLS client auth.

```python
nemoguardrails.library.hf_classifier.backends._check_cached(
    cache_key: str,
    model_name: str
) -> typing.Any
```

Return cached pipeline, wait on loading, or raise on error. Returns None on miss.

```python
nemoguardrails.library.hf_classifier.backends._flatten_ints(
    nested: typing.Any
) -> typing.List[int]
```

```python
nemoguardrails.library.hf_classifier.backends._get_or_create_pipeline(
    model_name: str,
    task: str,
    parameters: typing.Dict[str, typing.Any]
) -> typing.Any
```

```python
nemoguardrails.library.hf_classifier.backends._get_timeout(
    config: nemoguardrails.rails.llm.config.RemoteHFClassifierConfig
) -> httpx.Timeout
```

```python
nemoguardrails.library.hf_classifier.backends._pipeline_cache_key(
    model_name: str,
    task: str,
    parameters: typing.Dict[str, typing.Any]
) -> str
```

```python
nemoguardrails.library.hf_classifier.backends.get_backend(
    config: nemoguardrails.rails.llm.config.HFClassifierConfig,
    name: str = ''
) -> nemoguardrails.library.hf_classifier.backends.ClassifierBackend
```

Get or create a cached backend instance from classifier config.

```python
nemoguardrails.library.hf_classifier.backends._BACKENDS = {'local': LocalBackend, 'vllm': VLLMBackend, 'kserve': KServeBackend, 'fms': FMS...
```

```python
nemoguardrails.library.hf_classifier.backends._DEFAULT_TIMEOUT = 30.0
```

```python
nemoguardrails.library.hf_classifier.backends._FAILED_LOAD_MSG = "HF pipeline '{model}' failed to load at startup: {error}. Check server logs for...
```

```python
nemoguardrails.library.hf_classifier.backends._HTTP_ONLY_PARAMS = frozenset({'timeout', 'verify_ssl', 'ca_cert', 'client_cert', 'client_key'})
```

```python
nemoguardrails.library.hf_classifier.backends._TRANSIENT_ERRORS = (httpx.NetworkError, httpx.TimeoutException, httpx.RemoteProtocolError, OSError,...
```

```python
nemoguardrails.library.hf_classifier.backends._backend_instances: Dict[str, ClassifierBackend] = {}
```

```python
nemoguardrails.library.hf_classifier.backends._pipelines: Dict[str, Any] = {}
```

```python
nemoguardrails.library.hf_classifier.backends._pipelines_lock = threading.Lock()
```

```python
nemoguardrails.library.hf_classifier.backends._ssl_cache: Dict[tuple, _HTTPXSSLConfig] = {}
```

```python
nemoguardrails.library.hf_classifier.backends._warned_env_vars: set = set()
```

```python
nemoguardrails.library.hf_classifier.backends.log = logging.getLogger(__name__)
```