ReferencePython SDK ReferenceNemoguardrailsIntegrationsLangchainProvidersHuggingface

nemoguardrails.integrations.langchain.providers.huggingface.streamers

View as Markdown

Module Contents

Classes

NameDescription
AsyncTextIteratorStreamerSimple async implementation for HuggingFace Transformers streamers.
TextStreamer-

Data

TRANSFORMERS_AVAILABLE

API

class nemoguardrails.integrations.langchain.providers.huggingface.streamers.AsyncTextIteratorStreamer(
    tokenizer: transformers.AutoTokenizer,
    skip_prompt: bool = False,
    decode_kwargs = {}
)

Bases: TextStreamer

Simple async implementation for HuggingFace Transformers streamers.

This follows closely how transformers.generation.streamers.TextIteratorStreamer works, with minor modifications to make it async.

loop
Optional[AbstractEventLoop] = None
text_queue
Queue[str] = asyncio.Queue()
nemoguardrails.integrations.langchain.providers.huggingface.streamers.AsyncTextIteratorStreamer.__aiter__()
nemoguardrails.integrations.langchain.providers.huggingface.streamers.AsyncTextIteratorStreamer.__anext__()
async
nemoguardrails.integrations.langchain.providers.huggingface.streamers.AsyncTextIteratorStreamer.on_finalized_text(
    text: str,
    stream_end: bool = False
)

Put the new text in the queue. If the stream is ending, also put a stop signal in the queue.

class nemoguardrails.integrations.langchain.providers.huggingface.streamers.TextStreamer(
    args = (),
    kwargs = {}
)
 
nemoguardrails.integrations.langchain.providers.huggingface.streamers.TRANSFORMERS_AVAILABLE = True