bridge.models.qwen.qwen3_bridge

Module Contents

Classes

Qwen3Bridge

Megatron Bridge for Qwen3 Causal LM.

API

class bridge.models.qwen.qwen3_bridge.Qwen3Bridge

Bases: megatron.bridge.models.conversion.model_bridge.MegatronModelBridge

Megatron Bridge for Qwen3 Causal LM.

This bridge handles the conversion between HuggingFace Qwen3ForCausalLM and Megatron-Core GPTModel formats. Qwen3 differs from Qwen2 by using QK layernorm and no QKV bias.

.. rubric:: Example

from megatron.bridge import AutoBridge bridge = AutoBridge.from_hf_pretrained(“Qwen/Qwen3-1.7B”) model_config = bridge.get_model_config()

MODEL_CONFIG_CLASS

None

USE_MODEL_CONFIG_FOR_CONVERSION

True

hf_config_to_model_config_kwargs(

hf_config: transformers.PretrainedConfig,

) → dict[str, Any]

Convert a Hugging Face Qwen3 config to builder config kwargs.

hf_config_to_provider_kwargs(

hf_config: transformers.PretrainedConfig,

) → dict[str, Any]

Adapt the canonical builder mapping to the deprecated provider path.

mapping_registry() → megatron.bridge.models.conversion.mapping_registry.MegatronMappingRegistry

Return the MegatronMappingRegistry for Qwen3 parameter conversion.

Covers all Megatron-Core parameter names for both the standard decoder layers and the MTP (Multi-Token Prediction) transformer layers that are present when mtp_num_layers >= 1.

Simple 1:1 renames are expressed as :class:AutoMapping entries. The fused QKV matrix is handled by :class:QKVMapping and the gated MLP gate+up projection by :class:GatedMLPMapping.