bridge.models.olmoe.olmoe_bridge#

Module Contents#

Classes#

OlMoEBridge

Megatron Bridge for OlMoE Models.

Data#

logger

API#

bridge.models.olmoe.olmoe_bridge.logger#

‘getLogger(…)’

class bridge.models.olmoe.olmoe_bridge.OlMoEBridge#

Bases: megatron.bridge.models.conversion.model_bridge.MegatronModelBridge

Megatron Bridge for OlMoE Models.

This bridge handles the conversion between HuggingFace OlMoEForCausalLM and Megatron-Core GPTModel formats. OlMoE models use mixture of experts architecture with QK layernorm.

.. rubric:: Example

from megatron.bridge import AutoBridge bridge = AutoBridge.from_hf_pretrained(“allenai/OLMoE-1B-7B-0125”) model_config = bridge.get_model_config()

MODEL_CONFIG_CLASS#

None

USE_MODEL_CONFIG_FOR_CONVERSION#

True

hf_config_to_model_config_kwargs(
hf_config: transformers.PretrainedConfig,
) → dict[str, Any]#

Convert a Hugging Face OlMoE config to builder config kwargs.

OlMoE uses QK layernorm and mixture of experts.

hf_config_to_provider_kwargs(
hf_config: transformers.PretrainedConfig,
) → dict[str, Any]#

Adapt the canonical builder mapping to the deprecated provider path.

mapping_registry() → megatron.bridge.models.conversion.mapping_registry.MegatronMappingRegistry#