> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/guardrails/llms.txt.
> For full documentation content, see https://docs.nvidia.com/nemo/guardrails/llms-full.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/guardrails/_mcp/server.

# nemoguardrails.streaming

## Module Contents

### Classes

| Name                                                             | Description                                                           |
| ---------------------------------------------------------------- | --------------------------------------------------------------------- |
| [`StreamingHandler`](#nemoguardrails-streaming-StreamingHandler) | Provider-agnostic streaming handler with prefix/suffix/stop handling. |

### Data

[`END_OF_STREAM`](#nemoguardrails-streaming-END_OF_STREAM)

[`log`](#nemoguardrails-streaming-log)

### API

```python
class nemoguardrails.streaming.StreamingHandler(
    enable_print: bool = False,
    enable_buffer: bool = False,
    include_metadata: typing.Optional[bool] = False,
    include_generation_metadata: typing.Optional[bool] = None
)
```

**Bases:** `AsyncIterator`

Provider-agnostic streaming handler with prefix/suffix/stop handling.

Implements AsyncIterator interface so it can be used directly to stream
back the response. Chunks are pushed via push\_chunk() and consumed via
async iteration.

```python
nemoguardrails.streaming.StreamingHandler.__aiter__()
```

```python
nemoguardrails.streaming.StreamingHandler.__anext__()
```

async

```python
nemoguardrails.streaming.StreamingHandler._process(
    chunk: typing.Union[str, object],
    metadata: typing.Optional[typing.Dict[str, typing.Any]] = None
)
```

async

Process a chunk of text.

If we're in buffering mode, record the text.
Otherwise, update the full completion, check for stop tokens, and enqueue the chunk.

```python
nemoguardrails.streaming.StreamingHandler.disable_buffering()
```

async

When we disable the buffer, we process the buffer as a chunk.

```python
nemoguardrails.streaming.StreamingHandler.enable_buffering()
```

async

```python
nemoguardrails.streaming.StreamingHandler.finish()
```

async

Signal end of stream.

```python
nemoguardrails.streaming.StreamingHandler.push_chunk(
    chunk: typing.Union[str, None],
    metadata: typing.Optional[typing.Dict[str, typing.Any]] = None
)
```

async

Push a new string chunk to the stream.

**Parameters:**

String chunk to push, None to signal end of stream, or END\_OF\_STREAM sentinel.

Optional metadata about the generation.

```python
nemoguardrails.streaming.StreamingHandler.set_pattern(
    prefix: typing.Optional[str] = None,
    suffix: typing.Optional[str] = None
)
```

Sets the pattern that is expected.

If a prefix or a suffix are specified, they will be removed from the output.

```python
nemoguardrails.streaming.StreamingHandler.set_pipe_to(
    another_handler: nemoguardrails.streaming.StreamingHandler
)
```

```python
nemoguardrails.streaming.StreamingHandler.wait()
```

async

Waits until the stream finishes and returns the full completion.

```python
nemoguardrails.streaming.StreamingHandler.wait_top_k_nonempty_lines(
    k: int
)
```

async

Waits for top k non-empty lines from the LLM.

When k lines have been received (and k+1 has been started) it will return
and remove them from the buffer.

```python
nemoguardrails.streaming.END_OF_STREAM = object()
```

```python
nemoguardrails.streaming.log = logging.getLogger(__name__)
```