Skip to main content
Ctrl+K
Megatron Bridge - Home Megatron Bridge - Home

Megatron Bridge

  • GitHub
Megatron Bridge - Home Megatron Bridge - Home

Megatron Bridge
  • GitHub

Table of Contents

Guides

  • Parallelisms Guide
  • Performance
  • Performance Tuning Guide
  • Using Recipes
  • NeMo 2.0 to Megatron Bridge Migration Guide
  • Megatron-LM to Megatron Bridge Guide

Bridge with 🤗 Hugging Face

  • Get Started with 🤗 Hugging Face Conversion
  • Megatron Bridge Conversion Technical Details

Supported Models

  • Large Language Models
    • Llama 3
    • Qwen
  • Vision Language Models
    • Nemotron Nano V2 VL

Training and Customization

  • Configuration Overview
  • Training Entry Points
  • Training Loop Configuration
  • Optimizer and Scheduler Configuration
  • Logging and Monitoring
  • Profiling
  • Checkpointing
  • Resiliency
  • Mixed Precision Training
  • Communication Overlap
  • Attention Optimizations
  • Activation Recomputation
  • CPU Offloading
  • Parameter-Efficient Fine-Tuning (PEFT)
  • Packed Sequences

Development

  • Contribute a New Model to Megatron Bridge
  • Adapting Megatron Bridge in Reinforcement Learning Frameworks
  • Documentation Development
  • API Reference
    • bridge
      • bridge.peft
        • bridge.peft.dora_layers
        • bridge.peft.lora
        • bridge.peft.dora
        • bridge.peft.adapter_wrapper
        • bridge.peft.walk_utils
        • bridge.peft.base
        • bridge.peft.module_matcher
        • bridge.peft.lora_layers
        • bridge.peft.canonical_lora
        • bridge.peft.utils
      • bridge.data
        • bridge.data.vlm_datasets
        • bridge.data.datasets
        • bridge.data.builders
        • bridge.data.hf_processors
        • bridge.data.loaders
        • bridge.data.samplers
        • bridge.data.finetuning
        • bridge.data.utils
      • bridge.models
        • bridge.models.gpt_oss
        • bridge.models.conversion
        • bridge.models.qwen
        • bridge.models.gemma
        • bridge.models.nemotron
        • bridge.models.qwen_vl
        • bridge.models.llama
        • bridge.models.olmoe
        • bridge.models.mamba
        • bridge.models.llama_nemotron
        • bridge.models.gemma_vl
        • bridge.models.glm
        • bridge.models.mistral
        • bridge.models.deepseek
        • bridge.models.decorators
        • bridge.models.nemotronh
        • bridge.models.hf_pretrained
        • bridge.models.gpt_full_te_layer_autocast_spec
        • bridge.models.model_provider
        • bridge.models.config
        • bridge.models.gpt_provider
        • bridge.models.transformer_config
        • bridge.models.t5_provider
      • bridge.recipes
        • bridge.recipes.gpt_oss
        • bridge.recipes.qwen
        • bridge.recipes.gpt
        • bridge.recipes.gemma
        • bridge.recipes.qwen_vl
        • bridge.recipes.llama
        • bridge.recipes.mamba
        • bridge.recipes.utils
        • bridge.recipes.deepseek
        • bridge.recipes.nemotronh
        • bridge.recipes.run_plugins
      • bridge.utils
        • bridge.utils.vocab_utils
        • bridge.utils.instantiate_utils
        • bridge.utils.common_utils
        • bridge.utils.fusions
        • bridge.utils.yaml_utils
        • bridge.utils.import_utils
        • bridge.utils.decorators
      • bridge.training
        • bridge.training.tokenizers
        • bridge.training.mlm_compat
        • bridge.training.utils
        • bridge.training.model_load_save
        • bridge.training.fault_tolerance
        • bridge.training.pretrain
        • bridge.training.optim
        • bridge.training.state
        • bridge.training.train
        • bridge.training.nvrx_straggler
        • bridge.training.losses
        • bridge.training.eval
        • bridge.training.setup
        • bridge.training.initialize
        • bridge.training.checkpointing
        • bridge.training.inprocess_restart
        • bridge.training.config
        • bridge.training.comm_overlap
        • bridge.training.mixed_precision
        • bridge.training.finetune
        • bridge.training.deepep
        • bridge.training.profiling
        • bridge.training.gpt_step
        • bridge.training.vlm_step
        • bridge.training.forward_step_func_types
      • bridge.package_info
  • API Reference
  • bridge
  • bridge.recipes
  • bridge.recipes.nemotronh

bridge.recipes.nemotronh#

Submodules#

  • bridge.recipes.nemotronh.nemotronh
  • bridge.recipes.nemotronh.nemotron_nano_v2

Package Contents#

Data#

__all__

API#

bridge.recipes.nemotronh.__all__#

[‘nemotronh_4b_pretrain_config’, ‘nemotronh_8b_pretrain_config’, ‘nemotronh_47b_pretrain_config’, ‘n…

previous

bridge.recipes.deepseek.deepseek_v2

next

bridge.recipes.nemotronh.nemotronh
On this page
  • Submodules
  • Package Contents
    • Data
    • API
      • __all__
NVIDIA NVIDIA
Privacy Policy | Manage My Privacy | Do Not Sell or Share My Data | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2025, NVIDIA Corporation.