Supported Models

Megatron Core supports a wide range of language and multimodal models with optimized implementations for large-scale training.

Model Conversion

For converting HuggingFace models to Megatron format, use Megatron Bridge, the official standalone converter. Megatron Bridge supports an extensive list of models including LLaMA, Mistral, Mixtral, Qwen, DeepSeek, Gemma, Phi, Nemotron, and many more.

See the Megatron Bridge supported models list for the complete and up-to-date list of supported models.

• Language Models
• Multimodal Models
• Llama, Mistral and other Llama-like model support in Megatron-LM
• Contents
• Llama-2
• Llama-3.x
• Mistral-7b
• Other Llama-like model support
• Known numerical differences
• Using legacy model format