`bridge.models.qwen_vl.data.collate_fn`#

Qwen VL collator implementations.

Module Contents#

Classes#

QwenVLPreparedSequence

One fully processed, unpadded Qwen-VL sequence and its visual payload.

Functions#

`_normalize_qwen_video_paths`	Map path-based video parts to the inline schema expected by Qwen processors.
`_prepare_qwen_vl_sequence`	Process one example into the exact row consumed by THD packing.
`prepare_qwen_vl_sequence`	Process one Qwen-VL example once for online or collate-time packing.
`build_qwen_vl_packed_batch`	Build one canonical MCore THD row from prepared Qwen-VL sequences.
`qwen2_5_collate_fn`	Collate function for Qwen2.5 VL model.

Data#

`MISSING_QWEN_VL_UTILS_MSG`
`QWEN_VL_MIN_PIXELS`
`QWEN_VL_MAX_PIXELS`
`CHATML_ASSISTANT_START`
`CHATML_ASSISTANT_END`
`CHATML_OTHER_ROLE_STARTS`
`QWEN_VISUAL_KEYS`

API#

bridge.models.qwen_vl.data.collate_fn.MISSING_QWEN_VL_UTILS_MSG#: ‘qwen_vl_utils is required for Qwen2.5 VL processing. Please pip install qwen-vl-utils or provide c…’

bridge.models.qwen_vl.data.collate_fn.QWEN_VL_MIN_PIXELS#: 200704

bridge.models.qwen_vl.data.collate_fn.QWEN_VL_MAX_PIXELS#: 1003520

bridge.models.qwen_vl.data.collate_fn.CHATML_ASSISTANT_START#: ‘<|im_start|>assistant\n’

bridge.models.qwen_vl.data.collate_fn.CHATML_ASSISTANT_END#: ‘<|im_end|>\n’

bridge.models.qwen_vl.data.collate_fn.CHATML_OTHER_ROLE_STARTS#: None

bridge.models.qwen_vl.data.collate_fn.QWEN_VISUAL_KEYS#: ()

class bridge.models.qwen_vl.data.collate_fn.QwenVLPreparedSequence#

One fully processed, unpadded Qwen-VL sequence and its visual payload.

.. note::

This model-owned storage layout is provisional and may evolve as more VLMs adopt native packing. Callers should use sequence_length when they do not need Qwen-specific row or visual_values payloads.

row: dict[str, torch.Tensor]#: None

visual_values: dict[str, torch.Tensor]#: None

property sequence_length: int#: Return the exact post-processor token length.

bridge.models.qwen_vl.data.collate_fn._normalize_qwen_video_paths( example: dict[str, Any], ) → dict[str, Any]#: Map path-based video parts to the inline schema expected by Qwen processors.

bridge.models.qwen_vl.data.collate_fn._prepare_qwen_vl_sequence( example: dict[str, Any], processor: Any, *, text: str, images: list[Any], videos: list[Any], skipped_tokens: torch.Tensor, boundary_config: Any, min_pixels: int | None, max_pixels: int | None, require_assistant_matches: bool, ) → bridge.models.qwen_vl.data.collate_fn.QwenVLPreparedSequence#: Process one example into the exact row consumed by THD packing.

bridge.models.qwen_vl.data.collate_fn.prepare_qwen_vl_sequence( example: dict[str, Any], processor: Any, *, min_pixels: int | None = QWEN_VL_MIN_PIXELS, max_pixels: int | None = QWEN_VL_MAX_PIXELS, require_assistant_matches: bool = False, ) → bridge.models.qwen_vl.data.collate_fn.QwenVLPreparedSequence#: Process one Qwen-VL example once for online or collate-time packing.

bridge.models.qwen_vl.data.collate_fn.build_qwen_vl_packed_batch( sequences: list[bridge.models.qwen_vl.data.collate_fn.QwenVLPreparedSequence], *, sequence_length: int | None, pad_to_multiple_of: int, pad_to_max_length: bool = False, ) → dict[str, Any]#

Build one canonical MCore THD row from prepared Qwen-VL sequences.

When pad_to_max_length is true, the last physical segment is padded so the row reaches sequence_length while the unpadded THD boundaries keep describing only real tokens.

bridge.models.qwen_vl.data.collate_fn.qwen2_5_collate_fn( examples: list, processor, min_pixels: int | None = QWEN_VL_MIN_PIXELS, max_pixels: int | None = QWEN_VL_MAX_PIXELS, visual_keys: object = None, require_assistant_matches: bool = False, sequence_length: int | None = None, pad_to_max_length: bool = False, pad_to_multiple_of: int = 128, enable_in_batch_packing: bool = False, in_batch_packing_pad_to_multiple_of: int = 1, ) → dict[str, torch.Tensor]#: Collate function for Qwen2.5 VL model.

bridge.models.qwen_vl.data.collate_fn#