Skip to navigation

nemoguardrails.integrations.langchain.providers.huggingface

View as MarkdownOpen in Claude

Submodules

Package Contents

Classes

NameDescription
AsyncTextIteratorStreamerSimple async implementation for HuggingFace Transformers streamers.
HuggingFacePipelineCompatibleHackish way to add backward-compatibility functions to the Langchain class.

API

class nemoguardrails.integrations.langchain.providers.huggingface.AsyncTextIteratorStreamer(
tokenizer: transformers.AutoTokenizer,
skip_prompt: bool = False,
decode_kwargs = {}
)

Bases: TextStreamer

Simple async implementation for HuggingFace Transformers streamers.

This follows closely how transformers.generation.streamers.TextIteratorStreamer works, with minor modifications to make it async.

loop
Optional[AbstractEventLoop] = None
text_queue
Queue[str] = asyncio.Queue()
nemoguardrails.integrations.langchain.providers.huggingface.AsyncTextIteratorStreamer.__aiter__()
nemoguardrails.integrations.langchain.providers.huggingface.AsyncTextIteratorStreamer.__anext__()
async
nemoguardrails.integrations.langchain.providers.huggingface.AsyncTextIteratorStreamer.on_finalized_text(
text: str,
stream_end: bool = False
)

Put the new text in the queue. If the stream is ending, also put a stop signal in the queue.

class nemoguardrails.integrations.langchain.providers.huggingface.HuggingFacePipelineCompatible()

Bases: HuggingFacePipeline

Hackish way to add backward-compatibility functions to the Langchain class. TODO: Planning to add this fix directly to Langchain repo.

nemoguardrails.integrations.langchain.providers.huggingface.HuggingFacePipelineCompatible._acall(
prompt: str,
stop: typing.Optional[typing.List[str]] = None,
run_manager: typing.Optional[langchain_core.callbacks.manager.AsyncCallbackManagerForLLMRun] = None,
kwargs: typing.Any = {}
) -> str
async

Hackish way to add async support

nemoguardrails.integrations.langchain.providers.huggingface.HuggingFacePipelineCompatible._call(
prompt: str,
stop: typing.Optional[typing.List[str]] = None,
run_manager: typing.Optional[langchain_core.callbacks.manager.CallbackManagerForLLMRun] = None,
kwargs: typing.Any = {}
) -> str

Hackish way to perform a single llm call since Langchain dropped support