bridge.models.sarvam.sarvam_moe_bridge

Module Contents

Classes

SarvamMoEBridge

Megatron Hub Bridge for Sarvam MoE Causal LM.

API

class bridge.models.sarvam.sarvam_moe_bridge.SarvamMoEBridge

Bases: megatron.bridge.models.conversion.model_bridge.MegatronModelBridge

Megatron Hub Bridge for Sarvam MoE Causal LM.

This bridge handles the conversion between HuggingFace SarvamMoEForCausalLM and Megatron-Core GPTModel formats. Sarvam MoE models use mixture of experts architecture with QKV layernorm.

provider_bridge(

hf_pretrained: megatron.bridge.models.hf_pretrained.causal_lm.PreTrainedCausalLM,

) → megatron.bridge.models.sarvam.sarvam_provider.SarvamMoEModelProvider

mapping_registry() → megatron.bridge.models.conversion.mapping_registry.MegatronMappingRegistry