bridge.models.inkling.data.collate_fn#

Inkling2 image-text SFT collator.

Module Contents#

Classes#

Inkling2VisualInputs

Inkling2 image inputs: pixel_values [patches, 2, 40, 40, 3] and image_grid_thw [images, 3].

Functions#

inkling2_collate_fn

Collate Inkling2 conversations with images into a padded training batch.

Data#

API#

class bridge.models.inkling.data.collate_fn.Inkling2VisualInputs#

Bases: megatron.bridge.training.utils.visual_inputs.GenericVisualInputs

Inkling2 image inputs: pixel_values [patches, 2, 40, 40, 3] and image_grid_thw [images, 3].

The patches are five-dimensional but already flat over images, so they pass to the model unchanged instead of being reshaped as a batch of per-sample image stacks.

normalized_for_model() → dict[str, torch.Tensor]#

Return the non-None fields unchanged.

bridge.models.inkling.data.collate_fn.inkling2_collate_fn(
examples: list[dict[str, Any]],
processor: megatron.bridge.models.inkling.data.processor.Inkling2Processor,
*,
sequence_length: int | None = None,
pad_to_max_length: bool = False,
pad_to_multiple_of: int = 128,
enable_in_batch_packing: bool = False,
in_batch_packing_pad_to_multiple_of: int = 1,
) → dict[str, Any]#

Collate Inkling2 conversations with images into a padded training batch.

Parameters:
  • examples – Rows with a conversation message list, and optionally the tools and chat_template_kwargs (reasoning_effort) it is rendered with.

  • processor – The checkpoint’s :class:Inkling2Processor.

  • sequence_length – Longest sample, and the padded width with pad_to_max_length.

  • pad_to_max_length – Pad every batch to sequence_length, which keeps one compiled attention shape.

  • pad_to_multiple_of – Width multiple when not padding to sequence_length.

  • enable_in_batch_packing – Not supported; must be False.

  • in_batch_packing_pad_to_multiple_of – Unused without in-batch packing.

Returns:

input_ids, attention_mask, position_ids, labels and loss_mask over the assistant turns, and visual_inputs.

Raises:
  • TypeError – If processor is another class, such as Transformers’ processor of the same name.

  • ValueError – If in-batch packing is requested, or a row sets template arguments the chat template does not read.

bridge.models.inkling.data.collate_fn.__all__#

[‘Inkling2VisualInputs’, ‘inkling2_collate_fn’]