bridge.data.datasets.direct_sft

Runtime dataset for direct SFT examples.

Module Contents

Classes

DirectSFTDataset

Repeating wrapper over normalized SFT examples.

Functions

_collate_kwargs_for_impl

API

bridge.data.datasets.direct_sft._collate_kwargs_for_impl(

collate_impl: collections.abc.Callable[..., dict[str, torch.Tensor]],

collate_kwargs: dict[str, Any],

*,

require_packing_support: bool,

) → dict[str, Any]

class bridge.data.datasets.direct_sft.DirectSFTDataset(

base_examples: list[dict[str, Any]],

target_length: int,

processor: Any,

collate_impl: collections.abc.Callable[..., dict[str, torch.Tensor]] | None = None,

sequence_length: int | None = None,

pad_to_max_length: bool = False,

pad_to_multiple_of: int = 128,

enable_in_batch_packing: bool = False,

defer_in_batch_packing_to_step: bool = False,

in_batch_packing_pad_to_multiple_of: int = 1,

)

Bases: torch.utils.data.Dataset

Repeating wrapper over normalized SFT examples.

• Examples may use structured conversations or paired prompt-completion text, as selected by the owning dataset config. Optional modality fields are passed through and consumed by the collate function.

• Dataset length is set to a target length and indexes wrap around the underlying list to meet the requested size.

• A collate_fn attribute is exposed so the framework can pass it to the DataLoader.

Initialization

__len__() → int

__getitem__(idx: int) → dict[str, Any]