bridge.models.qwen_vl.modelling_qwen3_vl.rope#

Module Contents#

Classes#

Qwen3VLMoETextRotaryEmbedding

Qwen3-VL MoE text rotary position embedding.

Qwen3VLTextRotaryEmbedding

Qwen3-VL text rotary position embedding for non-MoE models.

API#

class bridge.models.qwen_vl.modelling_qwen3_vl.rope.Qwen3VLMoETextRotaryEmbedding#

Bases: transformers.models.qwen3_vl_moe.modeling_qwen3_vl_moe.Qwen3VLMoeTextRotaryEmbedding

Qwen3-VL MoE text rotary position embedding.

forward(
position_ids: torch.Tensor,
mrope_section: List[int],
) → torch.Tensor#

Forward pass of multimodal RoPE embedding.

Parameters:

  • position_ids (torch.Tensor) – A postion_id tensor with shape [3, batchsize, seqlens]

  • mrope_section (list[int]) – Multimodal rope section is for channel dimension of temporal, height and width in rope calculation.

Returns:

Raw frequency embeddings for Megatron Core (shape: [seq_length, bs, 1, dim]). Megatron Core will compute cos/sin internally and apply attention_scaling.

Return type:

Tensor

class bridge.models.qwen_vl.modelling_qwen3_vl.rope.Qwen3VLTextRotaryEmbedding#

Bases: bridge.models.qwen_vl.modelling_qwen3_vl.rope.Qwen3VLTextRotaryEmbedding

Qwen3-VL text rotary position embedding for non-MoE models.

forward(
position_ids: torch.Tensor,
mrope_section: List[int] = None,
) → torch.Tensor#

Forward pass for non-MoE Qwen3-VL RoPE.

Parameters:

  • position_ids – Position IDs tensor

  • mrope_section – Optional mrope section (if not provided, uses self.mrope_section)