bridge.models.hybrid_mla_provider#
Hybrid model provider for Multi-Latent Attention.
Module Contents#
Classes#
Provider for Hybrid models using Multi-Latent Attention. |
Functions#
Ensure shared DSA indexers never reference a previous pipeline stage. |
|
Add balanced pipeline separators before full-indexer DSA layers. |
|
Place DSA-safe pipeline separators in a Hybrid MLA layer pattern. |
API#
- bridge.models.hybrid_mla_provider._validate_dsa_pipeline_segments(
- main_pattern: str,
- *,
- topk_freq: int,
- skip_topk_offset: int,
Ensure shared DSA indexers never reference a previous pipeline stage.
- bridge.models.hybrid_mla_provider._add_dsa_pipeline_separators(
- pattern: str,
- *,
- num_pipeline_segments: int,
- topk_freq: int,
- skip_topk_offset: int,
Add balanced pipeline separators before full-indexer DSA layers.
num_pipeline_segmentscounts every model chunk: pipeline stages times virtual stages. Megatron-Core assigns segmentvp_stage * pp_size + pp_rankto each chunk.
- bridge.models.hybrid_mla_provider.apply_dsa_pipeline_separators(config: Any) None#
Place DSA-safe pipeline separators in a Hybrid MLA layer pattern.
Works on a
HybridMLAModelProvideror a builder-backed Hybrid model config, which exposes the nested transformer settings as flat attributes. Run it before the Hybrid finalization, which appends the MTP pattern and derivesnum_layers.- Parameters:
config – Provider or model config with a Hybrid layer pattern and DSA settings.
- Raises:
ValueError – If explicit first/last stage layer counts are set, or the pattern cannot be split into one segment per model chunk without crossing a DSA IndexShare group.
- class bridge.models.hybrid_mla_provider.HybridMLAModelProvider#
Bases:
megatron.bridge.models.hybrid.hybrid_provider.HybridModelProvider,megatron.bridge.models.transformer_config.MLATransformerConfigProvider for Hybrid models using Multi-Latent Attention.
- finalize() None#
Finalize the provider after adding DSA-safe pipeline segments.