bridge.models.deepseek.deepseek_v4_hybrid_provider

Provider for DeepSeek-V4 expressed as a Megatron-Core HybridModel.

DeepSeek-V4 is a Multi-Latent-Attention (MLA) model, so it needs every MLA configuration field (q_lora_rank, output_projection_groups, v_head_dim, rope_type / YaRN parameters, …). Those live on

class:

MLAModelProvider. It is also a hybrid model: each logical DeepSeek-V4 block is expressed as two Megatron hybrid layers — an attention-only layer (W/C/H symbol) and a MoE-only layer (E symbol) — driven by hybrid_layer_pattern and built by MCore’s standard hybrid stack. The hybrid provide()/finalize() logic lives on :class:HybridModelProvider.

This provider combines both. :class:HybridModelProvider is listed first so its provide() (which instantiates :class:~megatron.core.models.hybrid.hybrid_model.HybridModel) and finalize() (which derives num_layers from hybrid_layer_pattern) win over the GPT-model versions inherited via :class:MLAModelProvider, while issubclass(DeepSeekV4HybridModelProvider, MLAModelProvider) stays True so

meth:

MegatronModelBridge.provider_bridge keeps mapping the HF config through the direct MLA field names.

Module Contents

Classes

DeepSeekV4HybridModelProvider

MLA-capable :class:HybridModelProvider for DeepSeek-V4.

API

class bridge.models.deepseek.deepseek_v4_hybrid_provider.DeepSeekV4HybridModelProvider

Bases: megatron.bridge.models.hybrid.hybrid_provider.HybridModelProvider, megatron.bridge.models.mla_provider.MLAModelProvider

MLA-capable :class:HybridModelProvider for DeepSeek-V4.

All configuration is supplied by :class:DeepSeekV4Bridge.provider_bridge; this class only fixes the method-resolution order so a single provider is both an MLA config carrier and a hybrid-model builder.