Supported Models#

This directory contains family-organized documentation for models supported by Megatron Bridge. Each model page covers supported variants, Hugging Face <-> Megatron Bridge conversion, training recipe links, and model-specific notes.

Deprecation notice: DeepSeek V2/V2 Lite, Gemma 1 (2B/7B), Gemma 2 (2B/9B/27B), Llama 2, Llama Nemotron, Mistral 7B, Mistral Small 3 24B, Nemotron H v1 (4B/8B/47B/56B), Nemotron Nano v2 (9B/12B), Nemotron Nano v2 VL 12B, and the legacy NemotronForCausalLM bridge previously documented for Nemotron-4 340B are no longer actively maintained or tested against current upstream checkpoints. They will be removed in Megatron Bridge 0.7.0.

Family Index#

Family

Model documentation

Bailing

Ling 2.0

DeepSeek

DeepSeek V2 (deprecated), DeepSeek V3, DeepSeek V4

ERNIE

ERNIE 4.5 and ERNIE 4.5 VL

EXAONE

EXAONE 4, EXAONE 4.5 VL, and K-EXAONE

Falcon

Falcon H1

Gemma

Gemma (deprecated), Gemma 2 (deprecated), Gemma 3, Gemma 3 VL, Gemma 4 VL

GLM

GLM-4.5, GLM-4.5V, GLM-4.7 / GLM-4.7-Flash, GLM-5 / GLM-5.1 / GLM-5.2

GPT-OSS

GPT OSS

HY V3

HY V3

Kimi

Kimi K2, Kimi-K2.5-VL

Llama

Llama 2 (deprecated), Llama 3

MiniMax

MiniMax-M2 / M2.5 / M2.7, MiniMax-M3

Mistral

Mistral 7B / Small 3 24B (deprecated), Ministral 3

Xiaomi-MiMo

Xiaomi-MiMo

Moonlight

Moonlight

Nemotron

Llama Nemotron (deprecated), Nemotron H v1 and Nemotron Nano v2 (deprecated), Nemotron-3 Nano, Nemotron-3 Super, Nemotron Nano V2 VL (deprecated), Nemotron-3 Nano Omni

OLMoE

OLMoE

Qwen

Qwen, Qwen3-MoE, Qwen3-Next, Qwen2.5-VL, Qwen3-VL, Qwen3.5 / 3.6, Qwen2-Audio, Qwen2.5-Omni, Qwen3-Omni, Qwen3-ASR

Sarvam

Sarvam

StepFun

Step-3.5-Flash

Quick Navigation#

I want to#

Find model-specific docs -> Browse the family index above or use the navigation for the model’s family.

Convert models between formats -> See Bridge Guide for Hugging Face <-> Megatron conversion basics. Model pages include model-specific commands where available.

Get started with training -> See Training Documentation for training guides and Recipe Usage for pre-configured training recipes.

Add support for a new model -> Refer to Adding New Models.

Model Documentation Structure#

Each model documentation page typically includes:

  1. Model Overview - Architecture and key features

  2. Available Variants - Supported model sizes and configurations

  3. Conversion Examples - Converting between Hugging Face and Megatron formats

  4. Training Recipes - Links to training configurations and examples

  5. Architecture Details - Model-specific features and configurations

Model Support Overview#

Decoder-Only and Hybrid Backbones#

  • Bailing, DeepSeek, ERNIE, EXAONE, Falcon, Gemma, GLM, GPT-OSS, HY V3, Kimi, Llama, MiniMax, Mistral, Moonlight, Nemotron, OLMoE, Qwen, Sarvam, StepFun, and Xiaomi-MiMo

  • MoE and hybrid variants including Bailing, DeepSeek, GLM, GPT-OSS, MiniMax, Nemotron-3, OLMoE, Qwen3-MoE, Qwen3-Next, and Sarvam

Multimodal Variants#

  • Gemma 3 VL and Gemma 4 VL

  • ERNIE 4.5 VL and EXAONE 4.5 VL

  • GLM-4.5V

  • Kimi-K2.5-VL

  • Ministral 3

  • Nemotron Nano V2 VL (deprecated) and Nemotron-3 Nano Omni

  • Qwen2-Audio, Qwen2.5-VL, Qwen2.5-Omni, Qwen3-VL, Qwen3.5 / 3.6, Qwen3-Omni, and Qwen3-ASR

Conversion Support#

All model pages document support for one or both conversion directions:

  • Hugging Face -> Megatron Bridge: Load pretrained weights for training

  • Megatron Bridge -> Hugging Face: Export trained models for deployment

Conversion features:

  • Automatic architecture detection

  • Parallelism-aware conversion (TP/PP/VPP/CP/EP)

  • Streaming and memory-efficient transfers

  • Verification mechanisms for conversion accuracy

Refer to the Bridge Guide for detailed conversion instructions.