bridge.models.hybrid_mla_provider#

Hybrid model provider for Multi-Latent Attention.

Module Contents#

Classes#

HybridMLAModelProvider

Provider for Hybrid models using Multi-Latent Attention.

Functions#

_validate_dsa_pipeline_segments

Ensure shared DSA indexers never reference a previous pipeline stage.

_add_dsa_pipeline_separators

Add balanced pipeline separators before full-indexer DSA layers.

apply_dsa_pipeline_separators

Place DSA-safe pipeline separators in a Hybrid MLA layer pattern.

API#

bridge.models.hybrid_mla_provider._validate_dsa_pipeline_segments(
main_pattern: str,
*,
topk_freq: int,
skip_topk_offset: int,
) → None#

Ensure shared DSA indexers never reference a previous pipeline stage.

bridge.models.hybrid_mla_provider._add_dsa_pipeline_separators(
pattern: str,
*,
num_pipeline_segments: int,
topk_freq: int,
skip_topk_offset: int,
) → str#

Add balanced pipeline separators before full-indexer DSA layers.

num_pipeline_segments counts every model chunk: pipeline stages times virtual stages. Megatron-Core assigns segment vp_stage * pp_size + pp_rank to each chunk.

bridge.models.hybrid_mla_provider.apply_dsa_pipeline_separators(config: Any) → None#

Place DSA-safe pipeline separators in a Hybrid MLA layer pattern.

Works on a HybridMLAModelProvider or a builder-backed Hybrid model config, which exposes the nested transformer settings as flat attributes. Run it before the Hybrid finalization, which appends the MTP pattern and derives num_layers.

Parameters:

config – Provider or model config with a Hybrid layer pattern and DSA settings.

Raises:

ValueError – If explicit first/last stage layer counts are set, or the pattern cannot be split into one segment per model chunk without crossing a DSA IndexShare group.

class bridge.models.hybrid_mla_provider.HybridMLAModelProvider#

Bases: megatron.bridge.models.hybrid.hybrid_provider.HybridModelProvider, megatron.bridge.models.transformer_config.MLATransformerConfig

Provider for Hybrid models using Multi-Latent Attention.

finalize() → None#

Finalize the provider after adding DSA-safe pipeline segments.