`bridge.models.qwen_vl.data.energon`#

Module Contents#

Classes#

`QwenVLTaskSample`	HF-style Qwen VLM sample produced from an Energon `ChatMLSample`.
`QwenVLPackedTaskSample`	One Energon-selected group of prepared Qwen-VL samples.
`QwenVLTaskBatch`	Batched Qwen VLM tensors produced by the shared HF collate function.
`QwenVLTaskEncoder`	Energon task encoder for Qwen VL samples.

Functions#

`_stateless_with_failure_tolerance`	Apply Energon’s failure bound while keeping older container images importable.
`process_vision`	Minimal vision preprocessing wrapper using the provided processor (e.g., HF AutoProcessor).
`_resolve_hf_mm_token_ids`	Resolve HF tokenizer ids for and
`_visual_token_count`	Return merged visual token count for THW grid metadata.
`_aligned_sequence_length`	Return the physical length consumed by one aligned THD segment.
`convert_to_qwenvl_content`	Split user input into format QwenVL tokenizer accepts.

API#

bridge.models.qwen_vl.data.energon._stateless_with_failure_tolerance(failure_tolerance: int)#: Apply Energon’s failure bound while keeping older container images importable.

bridge.models.qwen_vl.data.energon.process_vision( processor, images, videos, fps=None, model_version: str = 'qwen-vl', min_pixels=None, max_pixels=None, )#: Minimal vision preprocessing wrapper using the provided processor (e.g., HF AutoProcessor).

bridge.models.qwen_vl.data.energon._resolve_hf_mm_token_ids(hf_tokenizer)#: Resolve HF tokenizer ids for and

bridge.models.qwen_vl.data.energon._visual_token_count(grid_thw: Any, merge_size: int) → int#: Return merged visual token count for THW grid metadata.

class bridge.models.qwen_vl.data.energon.QwenVLTaskSample#

HF-style Qwen VLM sample produced from an Energon ChatMLSample.

Expected input format: Produced by QwenVLTaskEncoder.encode_sample from an Energon ChatMLSample. Without native packing, example follows the HF VLM collate schema with inline image/video media parts. With native packing, prepared_sequence contains the eagerly processed model-owned sequence and example is None.

Output format: Consumed by QwenVLTaskEncoder.batch, which either collates the HF examples or assembles the prepared sequences into one packed THD row.

__key__: str#: None

__subflavors__: Dict#: None

example: Dict[str, Any] | None#: None

__restore_key__: tuple[Any, ...]#: ()

prepared_sequence: megatron.bridge.models.qwen_vl.data.collate_fn.QwenVLPreparedSequence | None#: None

class bridge.models.qwen_vl.data.energon.QwenVLPackedTaskSample#

One Energon-selected group of prepared Qwen-VL samples.

__key__: str#: None

__restore_key__: tuple[Any, ...]#: None

__subflavors__: Dict#: None

samples: List[bridge.models.qwen_vl.data.energon.QwenVLTaskSample]#: None

bridge.models.qwen_vl.data.energon._aligned_sequence_length( sequence: megatron.bridge.models.qwen_vl.data.collate_fn.QwenVLPreparedSequence, multiple: int, ) → int#: Return the physical length consumed by one aligned THD segment.

class bridge.models.qwen_vl.data.energon.QwenVLTaskBatch#

Bases: megatron.energon.Batch

Batched Qwen VLM tensors produced by the shared HF collate function.

__keys__: List[str]#: None

__subflavors__: List[Dict]#: None

input_ids: torch.Tensor#: None

position_ids: torch.Tensor#: None

labels: torch.Tensor#: None

loss_mask: torch.Tensor#: None

visual_inputs: megatron.bridge.training.utils.visual_inputs.GenericVisualInputs | None#: None

attention_mask: torch.Tensor | None#: None

cu_seqlens_q: torch.Tensor | None#: None

cu_seqlens_kv: torch.Tensor | None#: None

cu_seqlens_q_padded: torch.Tensor | None#: None

cu_seqlens_kv_padded: torch.Tensor | None#: None

max_seqlen_q: torch.Tensor | None#: None

max_seqlen_kv: torch.Tensor | None#: None

total_tokens: int | None#: None

bridge.models.qwen_vl.data.energon.convert_to_qwenvl_content( user_input: str, image_pattern: str = '<image>', video_pattern: str = '<video>', )#: Split user input into format QwenVL tokenizer accepts.

class bridge.models.qwen_vl.data.energon.QwenVLTaskEncoder( tokenizer, image_processor, temporal_patch_size: int = 2, spatial_merge_size: int = 2, patch_size: int = 14, max_padding_length: int = 4096, min_pixels: int = 200704, max_pixels: int = 1003520, max_num_images: int | None = 10, max_num_frames: int | None = 60, max_visual_tokens: int | None = 16384, pad_to_max_length: bool = False, pad_to_multiple_of: int = 128, enable_in_batch_packing: bool = False, in_batch_packing_pad_to_multiple_of: int = 1, enable_energon_packing: bool = False, )#

Bases: megatron.energon.DefaultTaskEncoder[megatron.bridge.data.energon.task_encoder_utils.ChatMLSample, bridge.models.qwen_vl.data.energon.QwenVLTaskSample, bridge.models.qwen_vl.data.energon.QwenVLTaskBatch, dict]

Energon task encoder for Qwen VL samples.

Parameters:

tokenizer – HF tokenizer for resolving multimodal token ids.
image_processor – HF processor passed to the Qwen collate function.
temporal_patch_size – Temporal patch size used for video token accounting.
spatial_merge_size – Spatial merge size used for visual token accounting.
patch_size – Vision patch size.
max_padding_length – Maximum sequence length accepted after collation.
min_pixels – Minimum pixel constraint forwarded to Qwen vision processing.
max_pixels – Maximum pixel constraint forwarded to Qwen vision processing.
max_num_images – Optional per-sample image count limit.
max_num_frames – Optional per-video frame limit.
max_visual_tokens – Optional per-sample visual token budget.
pad_to_max_length – Whether collate-time padding should pad non-packed batches to max_padding_length.
pad_to_multiple_of – Non-packed collate-time padding multiple used when pad_to_max_length is false.
enable_in_batch_packing – Whether the Qwen collate should do in-batch sequence packing.
in_batch_packing_pad_to_multiple_of – Per-sample padding multiple used only by the in-batch packed path, typically to satisfy CP/SP divisibility.
enable_energon_packing – Whether samples should be prepared for Energon’s native online packing hooks.

Initialization

encode_sample( sample: megatron.bridge.data.energon.task_encoder_utils.ChatMLSample, ) → bridge.models.qwen_vl.data.energon.QwenVLTaskSample#

Normalize one Energon sample into the HF-style Qwen collate schema.

Expected input format: sample is an Energon ChatMLSample with JSON conversation and optional decoded imgs / videos media payloads.

Output format: Without native packing, returns a QwenVLTaskSample containing the HF-style VLM example; model processing remains deferred to the Qwen collator. With native packing, the same model-owned processing runs eagerly and the result is stored in prepared_sequence while example is None.

select_samples_to_pack( samples: List[bridge.models.qwen_vl.data.energon.QwenVLTaskSample], ) → List[List[bridge.models.qwen_vl.data.energon.QwenVLTaskSample]]#: Partition one Energon buffer into packs bounded by seq_length.

pack_selected_samples( samples: List[bridge.models.qwen_vl.data.energon.QwenVLTaskSample], ) → bridge.models.qwen_vl.data.energon.QwenVLPackedTaskSample#: Wrap one selected group; Energon assigns its restore key afterwards.

collate_fn( examples: List[Dict[str, Any]], ) → dict[str, torch.Tensor]#

Collate Qwen HF-style examples with the shared HF dataset collator.

Expected input format: examples is a list of dictionaries in the same schema returned by HF-style VLM datasets: each item has conversation messages with multimodal content parts.

Output format: Returns the exact dictionary produced by qwen2_5_collate_fn: input_ids, labels, loss_mask, position_ids, optional attention_mask, and visual_inputs.

batch( samples: List[bridge.models.qwen_vl.data.energon.QwenVLTaskSample | bridge.models.qwen_vl.data.energon.QwenVLPackedTaskSample], ) → bridge.models.qwen_vl.data.energon.QwenVLTaskBatch#

Collate normalized Energon samples with the shared Qwen HF collator.

Expected input format: Either normalized QwenVLTaskSample objects whose example fields follow the HF VLM collate schema, or one physical QwenVLPackedTaskSample selected by Energon native packing.

Output format: Returns QwenVLTaskBatch carrying the same tensors as qwen2_5_collate_fn plus Energon batch metadata. Native packs may be padded to seq_length while retaining separate real and physical THD boundaries.

encode_batch( batch: bridge.models.qwen_vl.data.energon.QwenVLTaskBatch, ) → dict#: Encode batch in dict

bridge.models.qwen_vl.data.energon#

Module Contents#

Classes#

Functions#

API#

`bridge.models.qwen_vl.data.energon`#