Streaming Configuration

The NeMo Guardrails library supports streaming out of the box when using the stream_async() method. No configuration is required to enable basic streaming.

When you have output rails configured, you need to explicitly enable streaming for them to process tokens in chunked mode.

Quick Example

When using streaming with output rails:

1
rails:
2
  output:
3
    flows:
4
      - self check output
5
    streaming:
6
      enabled: True
7
      chunk_size: 200
8
      context_size: 50

Streaming Configuration Details

The following guides provide detailed documentation for streaming configuration.