OutputRailsStreamingConfig

class nemo_microservices.types.shared.OutputRailsStreamingConfig(*args: Any, **kwargs: Any)

Bases: BaseModel

chunk_size: int | None = None

The number of tokens in each processing chunk.

This is the size of the token block on which output rails are applied.

context_size: int | None = None

The number of tokens carried over from the previous chunk to provide context for continuity in processing.

enabled: bool | None = None

Enables streaming mode when True.

stream_first: bool | None = None

If True, token chunks are streamed immediately before output rails are applied.