nemoguardrails.library.hf_classifier.backends | NVIDIA NeMo Guardrails Library Developer Guide

Pluggable inference backends for HuggingFace classifier rails.

Module Contents

Classes

Name	Description
`ClassificationResult`	-
`ClassifierBackend`	Abstract interface for HuggingFace classifier inference.
`FMSBackend`	FMS guardrails-detectors `/api/v1/text/contents` backend.
`KServeBackend`	KServe v1 inference predict backend.
`LocalBackend`	Local HF Transformers pipeline backend.
`VLLMBackend`	vLLM `/classify` endpoint backend.
`_HTTPXSSLConfig`	Stores httpx-compatible SSL parameters (verify, cert).
`_PipelineLoadError`	Sentinel stored in _pipelines when a pipeline fails to load at startup.
`_PipelineLoading`	Sentinel stored while a pipeline is being loaded.
`_RemoteBackend`	Base class for remote HTTP classifier backends.

Functions

Name	Description
`_build_headers`	Build HTTP request headers from classifier config.
`_build_ssl_config`	Build httpx SSL params from config parameters (cached).
`_check_cached`	Return cached pipeline, wait on loading, or raise on error. Returns None on miss.
`_flatten_ints`	-
`_get_or_create_pipeline`	-
`_get_timeout`	-
`_pipeline_cache_key`	-
`get_backend`	Get or create a cached backend instance from classifier config.

Data

API

class nemoguardrails.library.hf_classifier.backends.ClassificationResult

Bases: typing.TypedDict

label

str

score

float

class nemoguardrails.library.hf_classifier.backends.ClassifierBackend()

Abstract

Bases: ABC

Abstract interface for HuggingFace classifier inference.

nemoguardrails.library.hf_classifier.backends.ClassifierBackend.classify(
    text: str
) -> typing.List[nemoguardrails.library.hf_classifier.backends.ClassificationResult]

asyncabstract

Classify a single text. Returns list of label/score detections.

nemoguardrails.library.hf_classifier.backends.ClassifierBackend.close() -> None

async

Release resources (e.g. HTTP sessions). Override in subclasses.

class nemoguardrails.library.hf_classifier.backends.FMSBackend(
    config: nemoguardrails.rails.llm.config.RemoteHFClassifierConfig
)

Bases: _RemoteBackend

FMS guardrails-detectors /api/v1/text/contents backend.

Tested against: FMS guardrails-detectors API v1. Expects response shape::

[[{“detection_type”: ”…”, “score”: float}, …]]

Raises ValueError if the response is not a list-of-lists or detection entries lack required keys.

_threshold

= config.threshold

_url

= config.base_url + '/api/v1/text/contents'

nemoguardrails.library.hf_classifier.backends.FMSBackend.classify(
    text: str
) -> typing.List[nemoguardrails.library.hf_classifier.backends.ClassificationResult]

async

class nemoguardrails.library.hf_classifier.backends.KServeBackend(
    config: nemoguardrails.rails.llm.config.RemoteHFClassifierConfig
)

Bases: _RemoteBackend

KServe v1 inference predict backend.

Tested against: KServe v1 predict API (/v1/models/{name}:predict). Handles three predictions shapes:

dict: class-index → probability (--return_probabilities)
int/float: argmax class index, score assumed 1.0
nested list: token-level class indices, flattened and deduplicated

Raises ValueError if predictions key is missing or the prediction has an unrecognised type.

_url

nemoguardrails.library.hf_classifier.backends.KServeBackend.classify(
    text: str
) -> typing.List[nemoguardrails.library.hf_classifier.backends.ClassificationResult]

async

class nemoguardrails.library.hf_classifier.backends.LocalBackend(
    config: nemoguardrails.rails.llm.config.LocalHFClassifierConfig
)

Bases: ClassifierBackend

Local HF Transformers pipeline backend.

Tested against: transformers >= 4.35 pipeline API for text-classification and token-classification tasks.

nemoguardrails.library.hf_classifier.backends.LocalBackend.classify(
    text: str
) -> typing.List[nemoguardrails.library.hf_classifier.backends.ClassificationResult]

async

class nemoguardrails.library.hf_classifier.backends.VLLMBackend(
    config: nemoguardrails.rails.llm.config.RemoteHFClassifierConfig
)

Bases: _RemoteBackend

vLLM /classify endpoint backend.

Tested against: vLLM v0.6.x /classify API. Expects response shape::

{“data”: [{“label”: ”…”, “probs”: [float, …]}]}

Raises ValueError if the response is missing required keys (data, label), indicating an API change.

_model_name

= config.model

_url

= config.base_url + '/classify'

nemoguardrails.library.hf_classifier.backends.VLLMBackend.classify(
    text: str
) -> typing.List[nemoguardrails.library.hf_classifier.backends.ClassificationResult]

async

class nemoguardrails.library.hf_classifier.backends._HTTPXSSLConfig(
    verify: typing.Union[str, bool] = True,
    cert: typing.Optional[tuple] = None
)

Stores httpx-compatible SSL parameters (verify, cert).

class nemoguardrails.library.hf_classifier.backends._PipelineLoadError(
    error: str
)

Sentinel stored in _pipelines when a pipeline fails to load at startup. Prevents silent retry — classify() raises immediately with the original error.

class nemoguardrails.library.hf_classifier.backends._PipelineLoading()

Sentinel stored while a pipeline is being loaded. Prevents duplicate concurrent loads — a second caller waits on the event.

_WAIT_TIMEOUT

= 600.0

error

Optional[BaseException] = None

ready

= threading.Event()

nemoguardrails.library.hf_classifier.backends._PipelineLoading.wait() -> typing.Any

class nemoguardrails.library.hf_classifier.backends._RemoteBackend(
    config: nemoguardrails.rails.llm.config.RemoteHFClassifierConfig
)

Bases: ClassifierBackend

Base class for remote HTTP classifier backends.

Uses a fresh httpx client per request — no connection pool state to manage. Classifier calls are infrequent and latency-tolerant (inference >> TLS handshake), so connection reuse provides negligible benefit while introducing stale-connection complexity.

_ssl

= _build_ssl_config(config)

_timeout

= _get_timeout(config)

nemoguardrails.library.hf_classifier.backends._RemoteBackend._post(
    url: str,
    json: typing.Dict[str, typing.Any]
) -> httpx.Response

async

POST with one automatic retry on transient failures.

nemoguardrails.library.hf_classifier.backends._build_headers(
    config: nemoguardrails.rails.llm.config.RemoteHFClassifierConfig
) -> typing.Dict[str, str]

Build HTTP request headers from classifier config.

nemoguardrails.library.hf_classifier.backends._build_ssl_config(
    config: nemoguardrails.rails.llm.config.RemoteHFClassifierConfig
) -> nemoguardrails.library.hf_classifier.backends._HTTPXSSLConfig

Build httpx SSL params from config parameters (cached).

Reads from config.parameters:

verify_ssl (bool, default True): set to False to skip TLS verification.
ca_cert (str): path to a CA bundle file for custom/internal CAs.
client_cert (str) + client_key (str): paths for mTLS client auth.

nemoguardrails.library.hf_classifier.backends._check_cached(
    cache_key: str,
    model_name: str
) -> typing.Any

Return cached pipeline, wait on loading, or raise on error. Returns None on miss.

nemoguardrails.library.hf_classifier.backends._flatten_ints(
    nested: typing.Any
) -> typing.List[int]

nemoguardrails.library.hf_classifier.backends._get_or_create_pipeline(
    model_name: str,
    task: str,
    parameters: typing.Dict[str, typing.Any]
) -> typing.Any

nemoguardrails.library.hf_classifier.backends._get_timeout(
    config: nemoguardrails.rails.llm.config.RemoteHFClassifierConfig
) -> httpx.Timeout

nemoguardrails.library.hf_classifier.backends._pipeline_cache_key(
    model_name: str,
    task: str,
    parameters: typing.Dict[str, typing.Any]
) -> str

nemoguardrails.library.hf_classifier.backends.get_backend(
    config: nemoguardrails.rails.llm.config.HFClassifierConfig,
    name: str = ''
) -> nemoguardrails.library.hf_classifier.backends.ClassifierBackend

Get or create a cached backend instance from classifier config.

nemoguardrails.library.hf_classifier.backends._BACKENDS = {'local': LocalBackend, 'vllm': VLLMBackend, 'kserve': KServeBackend, 'fms': FMS...

nemoguardrails.library.hf_classifier.backends._DEFAULT_TIMEOUT = 30.0

nemoguardrails.library.hf_classifier.backends._FAILED_LOAD_MSG = "HF pipeline '{model}' failed to load at startup: {error}. Check server logs for...

nemoguardrails.library.hf_classifier.backends._HTTP_ONLY_PARAMS = frozenset({'timeout', 'verify_ssl', 'ca_cert', 'client_cert', 'client_key'})

nemoguardrails.library.hf_classifier.backends._TRANSIENT_ERRORS = (httpx.NetworkError, httpx.TimeoutException, httpx.RemoteProtocolError, OSError,...

nemoguardrails.library.hf_classifier.backends._backend_instances: Dict[str, ClassifierBackend] = {}

nemoguardrails.library.hf_classifier.backends._pipelines: Dict[str, Any] = {}

nemoguardrails.library.hf_classifier.backends._pipelines_lock = threading.Lock()

nemoguardrails.library.hf_classifier.backends._ssl_cache: Dict[tuple, _HTTPXSSLConfig] = {}

nemoguardrails.library.hf_classifier.backends._warned_env_vars: set = set()

nemoguardrails.library.hf_classifier.backends.log = logging.getLogger(__name__)