bridge.models.inkling.data.collate_fn#
Inkling2 image-text SFT collator.
Module Contents#
Classes#
Inkling2 image inputs: |
Functions#
Collate Inkling2 conversations with images into a padded training batch. |
Data#
API#
- class bridge.models.inkling.data.collate_fn.Inkling2VisualInputs#
Bases:
megatron.bridge.training.utils.visual_inputs.GenericVisualInputsInkling2 image inputs:
pixel_values[patches, 2, 40, 40, 3]andimage_grid_thw[images, 3].The patches are five-dimensional but already flat over images, so they pass to the model unchanged instead of being reshaped as a batch of per-sample image stacks.
- normalized_for_model() dict[str, torch.Tensor]#
Return the non-None fields unchanged.
- bridge.models.inkling.data.collate_fn.inkling2_collate_fn(
- examples: list[dict[str, Any]],
- processor: megatron.bridge.models.inkling.data.processor.Inkling2Processor,
- *,
- sequence_length: int | None = None,
- pad_to_max_length: bool = False,
- pad_to_multiple_of: int = 128,
- enable_in_batch_packing: bool = False,
- in_batch_packing_pad_to_multiple_of: int = 1,
Collate Inkling2 conversations with images into a padded training batch.
- Parameters:
examples – Rows with a
conversationmessage list, and optionally thetoolsandchat_template_kwargs(reasoning_effort) it is rendered with.processor – The checkpoint’s :class:
Inkling2Processor.sequence_length – Longest sample, and the padded width with
pad_to_max_length.pad_to_max_length – Pad every batch to
sequence_length, which keeps one compiled attention shape.pad_to_multiple_of – Width multiple when not padding to
sequence_length.enable_in_batch_packing – Not supported; must be False.
in_batch_packing_pad_to_multiple_of – Unused without in-batch packing.
- Returns:
input_ids,attention_mask,position_ids,labelsandloss_maskover the assistant turns, andvisual_inputs.- Raises:
TypeError – If
processoris another class, such as Transformers’ processor of the same name.ValueError – If in-batch packing is requested, or a row sets template arguments the chat template does not read.
- bridge.models.inkling.data.collate_fn.__all__#
[‘Inkling2VisualInputs’, ‘inkling2_collate_fn’]