Skip to main content
Ctrl+K
Megatron Bridge - Home Megatron Bridge - Home

Megatron Bridge

  • GitHub
Megatron Bridge - Home Megatron Bridge - Home

Megatron Bridge
  • GitHub

Table of Contents

Performance

  • Performance
  • Performance Archive

Guides

  • Parallelisms Guide
  • Performance Tuning Guide
  • Using Recipes
  • NeMo 2.0 to Megatron Bridge Migration Guide
  • Megatron-LM to Megatron Bridge Guide

Bridge with 🤗 Hugging Face

  • Get Started with 🤗 Hugging Face Conversion
  • Megatron Bridge Conversion Technical Details

Supported Models

  • Supported Models
  • BAGEL-7B-MoT
  • Bailing
    • Ling 2.0
  • Beam
  • BERT
    • Megatron-Style BERT
  • DeepSeek
    • DeepSeek V3
    • DeepSeek V4
  • ERNIE 4.5
  • EXAONE
  • Falcon
    • Falcon H1
  • Gemma
    • Gemma 3
    • Gemma 3 VL (Vision-Language)
    • Gemma 4 VL (26B-A4B MoE)
    • DiffusionGemma
  • GLM
    • GLM 4.5
    • GLM-4.5V
    • GLM-4.7 and GLM-4.7-Flash
    • GLM-5
    • GLM-5.2
  • GPT-OSS
    • GPT OSS 20B
    • GPT OSS 120B
  • HY V3
  • Kimi
    • Kimi K2
    • Kimi-K2.5-VL
    • Kimi K3
  • Llama
    • Llama 3
  • MiniMax
    • MiniMax-M2
    • MiniMax-M3
  • Mistral
    • Ministral 3
  • Xiaomi-MiMo
    • Xiaomi-MiMo
  • Moonlight
    • Moonlight
  • Muse Glimmer 30B
  • Muse-Spark-Open-816B-A42B
  • Nemotron
    • Nemotron 3 Nano
    • Nemotron 3 Nano 4B
    • Nemotron-3 Nano Omni
    • Nemotron 3 Super
    • Nemotron 3 Ultra
    • Nemotron 3.5 Lightning
    • Nemotron 3.5 Super VL
    • Nemotron-3.5 Super Text-only
  • OLMoE
    • OLMoE
  • Qwen
    • Qwen3-235B-A22B
    • Qwen3-30B-A3B
    • Qwen3-8B
    • Qwen3-MoE
    • Qwen3-Next
    • Qwen2.5-VL
    • Qwen3-VL
    • Qwen3.6-35B-A3B
    • Qwen3.8-27B
    • Qwen2-Audio
    • Qwen2.5-Omni
    • Qwen3-Omni
    • Qwen3-ASR
  • Sarvam
    • Sarvam
  • StepFun
    • Step-3.5-Flash

Training and Customization

  • Configuration Overview
  • Training Entry Points
  • Data Preparation
  • Training Loop Configuration
  • Optimizer and Scheduler Configuration
  • Logging and Monitoring
  • Profiling
  • Theoretical Memory Estimator
  • Checkpointing
  • Megatron FSDP
  • Resiliency
  • Mixed Precision Training
  • CUDA Graphs
  • Hierarchical Context Parallel
  • Dynamic Context Parallel
  • Communication Overlap
  • Attention Optimizations
  • Activation Recomputation
  • CPU Offloading
  • MoE Training Optimization
  • Parameter-Efficient Fine-Tuning (PEFT)
  • Packed Sequences
  • MegatronMIMO Intra-Microbatch Reordering
  • Multi-Token Prediction (MTP)
  • DSpark Speculative Decoding (Draft Training) Design
  • Callbacks

Model Optimization with ModelOpt

  • Quantization
  • Knowledge Distillation
  • Pruning

Development

  • Contribute a New Model to Megatron Bridge
  • Adapting Megatron Bridge in Reinforcement Learning Frameworks
  • Documentation Development
  • API Reference
    • bridge
      • bridge.utils
        • bridge.utils.gdn_utils
        • bridge.utils.import_utils
        • bridge.utils.safe_url
        • bridge.utils.slurm_utils
        • bridge.utils.common_utils
        • bridge.utils.activation_map
        • bridge.utils.safe_pickle
        • bridge.utils.decorators
        • bridge.utils.fusions
        • bridge.utils.vocab_utils
        • bridge.utils.yaml_utils
        • bridge.utils.cuda_graph
        • bridge.utils.instantiate_utils
      • bridge.training
        • bridge.training.utils
        • bridge.training.post_training
        • bridge.training.tokenizers
        • bridge.training.mlm_compat
        • bridge.training.optim
        • bridge.training.vlm_step
        • bridge.training.pretrain
        • bridge.training.eval
        • bridge.training.nvrx_straggler
        • bridge.training.tensor_inspect
        • bridge.training.diffusion_gemma_step
        • bridge.training.callbacks
        • bridge.training.llava_step
        • bridge.training.megatron_mimo_step
        • bridge.training.dpo
        • bridge.training.checkpointing
        • bridge.training.gtp
        • bridge.training.distill
        • bridge.training.flex_dispatcher_backend
        • bridge.training.train_megatron_mimo
        • bridge.training.losses
        • bridge.training.config
        • bridge.training.pretrain_megatron_mimo
        • bridge.training.global_batch_packing
        • bridge.training.profiling
        • bridge.training.dpo_step
        • bridge.training.state
        • bridge.training.eval_context_parallel_rebinding
        • bridge.training.setup
        • bridge.training.audio_lm_step
        • bridge.training.initialize
        • bridge.training.forward_step_func_types
        • bridge.training.nemotron_omni_step
        • bridge.training.mixed_precision
        • bridge.training.train
        • bridge.training.inprocess_restart
        • bridge.training.megatron_mimo_parallel_utils
        • bridge.training.setup_megatron_mimo
        • bridge.training.fault_tolerance
        • bridge.training.finetune
        • bridge.training.dpo_train
        • bridge.training.model_load_save
        • bridge.training.comm_overlap
        • bridge.training.gpt_step
      • bridge.models
        • bridge.models.exaone
        • bridge.models.muse_spark_open
        • bridge.models.glm_moe_dsa
        • bridge.models.qwen
        • bridge.models.glm
        • bridge.models.glm_vl
        • bridge.models.mistral
        • bridge.models.nemotron_vl
        • bridge.models.decorators
        • bridge.models.gemma_vl
        • bridge.models.stepfun
        • bridge.models.megatron_mimo
        • bridge.models.qwen3_asr
        • bridge.models.gpt
        • bridge.models.bagel
        • bridge.models.ernie
        • bridge.models.ernie_vl
        • bridge.models.kimi_vl
        • bridge.models.mimo_v2_flash
        • bridge.models.kimi
        • bridge.models.gpt_oss
        • bridge.models.gemma
        • bridge.models.qwen_audio
        • bridge.models.mimo
        • bridge.models.hy_v3
        • bridge.models.beam
        • bridge.models.falcon_h1
        • bridge.models.hf_pretrained
        • bridge.models.hybrid
        • bridge.models.diffusion_gemma
        • bridge.models.sarvam
        • bridge.models.qwen_omni
        • bridge.models.nemotronh
        • bridge.models.bert
        • bridge.models.minimax_m3
        • bridge.models.common
        • bridge.models.qwen_vl
        • bridge.models.nemotron_omni
        • bridge.models.olmoe
        • bridge.models.deepseek
        • bridge.models.llama
        • bridge.models.conversion
        • bridge.models.muse_glimmer
        • bridge.models.minimax_m2
        • bridge.models.ministral3
        • bridge.models.mamba
        • bridge.models.bailing
        • bridge.models.gpt_full_te_layer_autocast_spec
        • bridge.models.mla_provider
        • bridge.models.gpt_provider
        • bridge.models.logit_dtype
        • bridge.models.model_provider
        • bridge.models.config
        • bridge.models.t5_provider
        • bridge.models.transformer_config
        • bridge.models.distillation_provider
      • bridge.data
        • bridge.data.megatron_mimo
        • bridge.data.energon
        • bridge.data.collators
        • bridge.data.builders
        • bridge.data.sources
        • bridge.data.packing
        • bridge.data.datasets
        • bridge.data.utils
        • bridge.data.loaders
        • bridge.data.sft_processing
        • bridge.data.batch_utils
        • bridge.data.conversation_processing
        • bridge.data.vlm_processing
        • bridge.data.base
        • bridge.data.token_utils
        • bridge.data.iterator_utils
        • bridge.data.samplers
      • bridge.diffusion
        • bridge.diffusion.base
        • bridge.diffusion.models
        • bridge.diffusion.data
        • bridge.diffusion.recipes
        • bridge.diffusion.common
        • bridge.diffusion.conversion
      • bridge.perf_recipes
        • bridge.perf_recipes.glm_moe_dsa
        • bridge.perf_recipes.qwen
        • bridge.perf_recipes.kimi
        • bridge.perf_recipes.gpt_oss
        • bridge.perf_recipes.wan
        • bridge.perf_recipes.nemotronh
        • bridge.perf_recipes.qwen_vl
        • bridge.perf_recipes.deepseek
        • bridge.perf_recipes.llama
        • bridge.perf_recipes.muse_glimmer
        • bridge.perf_recipes._common
        • bridge.perf_recipes.environment
      • bridge.recipes
        • bridge.recipes.exaone
        • bridge.recipes.muse_spark_open
        • bridge.recipes.flux
        • bridge.recipes.qwen
        • bridge.recipes.glm
        • bridge.recipes.glm_vl
        • bridge.recipes.stepfun
        • bridge.recipes.gpt
        • bridge.recipes.minimax
        • bridge.recipes.utils
        • bridge.recipes.gemma4_vl
        • bridge.recipes.bagel
        • bridge.recipes.kimi_vl
        • bridge.recipes.kimi
        • bridge.recipes.gpt_oss
        • bridge.recipes.gemma
        • bridge.recipes.nemotronh_multimodal
        • bridge.recipes.wan
        • bridge.recipes.gemma3_vl
        • bridge.recipes.beam
        • bridge.recipes.diffusion_gemma
        • bridge.recipes.qwen2_audio
        • bridge.recipes.moonlight
        • bridge.recipes.qwen_omni
        • bridge.recipes.nemotronh
        • bridge.recipes.qwen_vl
        • bridge.recipes.nemotron_omni
        • bridge.recipes.olmoe
        • bridge.recipes.deepseek
        • bridge.recipes.llama
        • bridge.recipes.muse_glimmer
        • bridge.recipes.ministral3
        • bridge.recipes.common
        • bridge.recipes.run_plugins
      • bridge.peft
        • bridge.peft.lora_layers
        • bridge.peft.dora_layers
        • bridge.peft.utils
        • bridge.peft.lora_merge
        • bridge.peft.module_matcher
        • bridge.peft.dora
        • bridge.peft.walk_utils
        • bridge.peft.adapter_wrapper
        • bridge.peft.lora
        • bridge.peft.canonical_lora
        • bridge.peft.recompute
        • bridge.peft.base
        • bridge.peft.multi_lora_layers
        • bridge.peft.multi_lora
      • bridge.package_info

Releases

  • Release Developer Guide
  • Cherrypick Policy
  • Software Component Versions
  • Changelog
  • Known Issues

Agent Skills

  • Agent Skills Reference
    • Build and Dependency
    • Bump Dependency
    • Testing
    • CI/CD
    • Review PR
    • MLM vs Bridge Training
    • Auto Recipe — Recipe Index & Recommendation
    • Adding New Model Support in Megatron-Bridge
    • LLM Bridge Patterns
    • VLM Bridge Patterns
    • Recipe Patterns
    • Test and Example Patterns
    • Create Model Verification Card
    • verl E2E Testing
    • NeMo-RL E2E Testing
    • CPU Offloading
    • MoE Long-Context Training
    • MoE Training Optimization Workflow
    • Nsight Systems Analysis for Megatron Bridge
    • MoE VLM Training
    • Memory Tuning
    • MoE Dispatcher Selection Guide
    • MoE Hardware Configuration Reference
    • Parallelism Strategy Selection Skill
    • CUDA Graphs
    • TP / DP / PP Communication Overlap Skill
    • Megatron FSDP Skill
    • Sequence Packing Skill
    • Hierarchical Context Parallel Skill
    • Vision DP Over CP Skill
    • MoE Expert-Parallel Overlap Skill
    • MoE Communication Overlap
    • Activation Recompute
    • Recipe Index (Library & Benchmark)
    • Nsight Systems analysis pitfalls
    • Nsight Systems SQLite recipes
    • Multi-Node Slurm
    • Multi-Node Slurm Templates
    • Memory Snapshot Analysis
    • Resiliency

Directory Readme Files

  • Megatron Bridge Documentation
  • Releases
  • Training and Customization
  • Model Optimization
  • Known Issues

Known Issues#

Known issues are maintained in the GitHub release notes. See the Known Issues section for your version, or check the latest release notes.

previous

Changelog

next

Agent Skills Reference
NVIDIA NVIDIA
Privacy Policy | Your Privacy Choices | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2026, NVIDIA Corporation.