bridge.models.glm_moe_dsa.glm5_bridge

Module Contents

Classes

GLM5Bridge

Megatron Bridge for the GLM-5 family (MoE + MLA + DSA).

Data

logger

API

bridge.models.glm_moe_dsa.glm5_bridge.logger

‘getLogger(…)’

class bridge.models.glm_moe_dsa.glm5_bridge.GLM5Bridge

Bases: megatron.bridge.models.conversion.model_bridge.MegatronModelBridge

Megatron Bridge for the GLM-5 family (MoE + MLA + DSA).

This bridge handles conversion between HuggingFace GlmMoeDsaForCausalLM and Megatron-Core HybridModel formats. zai-org/GLM-5, zai-org/GLM-5.1, zai-org/GLM-5.2, and zai-org/GLM-5.3 are auto-detected through this bridge, with version-specific DSA settings read from the HF config.

The architecture uses Multi-Latent Attention (MLA), Dynamic Sparse Attention (DSA) indexer layers, and Mixture-of-Experts (MoE). GLM-5.2/5.3 share the same architecture; GLM-5.3’s block-scaled FP8 checkpoint weights are dequantized to BF16 on import. This does not cover GLM-5.3-Flash, which uses a different architecture. See the model guides for checkpoint-specific verification and export requirements.

.. rubric:: Example

from megatron.bridge import AutoBridge bridge = AutoBridge.from_hf_pretrained(“zai-org/GLM-5.1”) provider = bridge.to_megatron_provider()

classmethod megatron_to_hf_config(

provider: megatron.bridge.models.glm_moe_dsa.glm5_provider.GLM5ModelProvider,

) → dict

Export HF block counts and preserve the reference model’s context length.

_should_map_hf_config_field(

hf_config: Any,

hf_name: str,

megatron_name: str,

value: Any,

) → bool

Return whether an HF config field should be mapped to the Megatron provider.

provider_bridge(

hf_pretrained: megatron.bridge.models.hf_pretrained.causal_lm.PreTrainedCausalLM,

) → megatron.bridge.models.glm_moe_dsa.glm5_provider.GLM5ModelProvider

generate_pipeline_layout(

num_layers: int,

pp: int,

mtp_layers: int = 0,

) → list[list[str]] | None

Defer DSA-aligned placement to the Hybrid provider’s pattern finalization.

The conversion launcher calls this hook before finalizing the provider. GPT decoder/embedding/MTP token layouts cannot describe GLM’s split D-/DE layers; GLM5ModelProvider.finalize builds their stage boundaries instead.

mapping_registry() → megatron.bridge.models.conversion.mapping_registry.MegatronMappingRegistry

maybe_modify_loaded_hf_weight(

hf_param: str | dict[str, str],

hf_state_dict: collections.abc.Mapping[str, torch.Tensor],

) → torch.Tensor | dict[str, torch.Tensor]

Dequantize block-scaled FP8 checkpoint weights during import.

static _maybe_dequantize_fp8(

weight: torch.Tensor,

param_name: str,

hf_state_dict: collections.abc.Mapping[str, torch.Tensor],

) → torch.Tensor