`bridge.data.collators.sequence_padding`#

Ordinary collate-time sequence padding and truncation helpers.

Module Contents#

`_ceil_to_multiple`
`use_processor_right_padding`	Temporarily force a processor tokenizer to use right padding.
`_token_key`
`_set_tokens`
`_pad_or_truncate_2d`
`_pad_or_truncate_position_ids`
`_pad_or_truncate_attention_mask`
`pad_or_truncate_sequence_batch`	Pad or truncate sequence-aligned tensors without applying packing.

bridge.data.collators.sequence_padding._ceil_to_multiple(value: int, multiple: int) → int#

bridge.data.collators.sequence_padding.use_processor_right_padding( processor: Any, ) → collections.abc.Iterator[None]#: Temporarily force a processor tokenizer to use right padding.

bridge.data.collators.sequence_padding._token_key( batch: collections.abc.MutableMapping[str, Any], ) → str#

bridge.data.collators.sequence_padding._set_tokens( batch: collections.abc.MutableMapping[str, Any], token_key: str, value: torch.Tensor, ) → None#

bridge.data.collators.sequence_padding._pad_or_truncate_2d( value: torch.Tensor | None, target_length: int, pad_value: int | float, ) → torch.Tensor | None#

bridge.data.collators.sequence_padding._pad_or_truncate_position_ids( position_ids: torch.Tensor | None, target_length: int, ) → torch.Tensor | None#

bridge.data.collators.sequence_padding._pad_or_truncate_attention_mask( attention_mask: torch.Tensor | None, target_length: int, ) → torch.Tensor | None#

bridge.data.collators.sequence_padding.pad_or_truncate_sequence_batch( batch: collections.abc.MutableMapping[str, Any], *, sequence_length: int | None, pad_to_max_length: bool = False, pad_to_multiple_of: int = 128, pad_token_id: int = 0, ignore_index: int = IGNORE_INDEX, sequence_tensor_pad_values: collections.abc.Mapping[str, int | float] | None = None, ) → None#: Pad or truncate sequence-aligned tensors without applying packing.