nemo_microservices.types.shared.output_rails_streaming_config#
Module Contents#
Classes#
API#
- class nemo_microservices.types.shared.output_rails_streaming_config.OutputRailsStreamingConfig(/, **data: Any)#
Bases:
nemo_microservices._models.BaseModel- chunk_size: int | None#
None
The number of tokens in each processing chunk.
This is the size of the token block on which output rails are applied.
- context_size: int | None#
None
The number of tokens carried over from the previous chunk to provide context for continuity in processing.
- enabled: bool | None#
None
Enables streaming mode when True.
- stream_first: bool | None#
None
If True, token chunks are streamed immediately before output rails are applied.