`nemo_microservices.types.shared.output_rails_streaming_config`#

Module Contents#

class nemo_microservices.types.shared.output_rails_streaming_config.OutputRailsStreamingConfig(/, **data: typing.Any)#

Bases: nemo_microservices._models.BaseModel

chunk_size: Optional[int]#

None

The number of tokens in each processing chunk.

This is the size of the token block on which output rails are applied.

context_size: Optional[int]#

None

The number of tokens carried over from the previous chunk to provide context for continuity in processing.

enabled: Optional[bool]#

None

Enables streaming mode when True.

stream_first: Optional[bool]#

None

If True, token chunks are streamed immediately before output rails are applied.