Skip to main content

Ctrl+K

Megatron Core

GitHub

Megatron Core

GitHub

Table of Contents

About Megatron Core

Overview
Release Notes and Roadmaps

Get Started

Installation
Your First Training Run

Basic Usage

Data Preparation
Training Examples
Parallelism Strategies Guide

Supported Models

Supported Models

Advanced Features

Deterministic Training
Mixture of Experts
Megatron-FSDP
Distributed Optimizer
Weighted Checkpoint Merge
Optimizer CPU Offload
Fine-Grained Activation Offloading
Data Loading at Scale
Megatron Energon
Megatron RL
Tokenizers
GDN Elementwise Fusion

Inference

Inference User Guide

Observability

Observability Overview
Configuration
Span Groups
Metrics
Pipeline-Parallel Trace Correlation
Extending Instrumentation

Developer Guide

Contributing to Megatron-LM
How to Submit a PR
Oncall Overview
Generating Docs Locally

API Reference

API Guide
API Reference
- core

Resources

Migrate from GPTModel to HybridModel
Discussions

API Guide
Internal Utilities

Is this page helpful?

Internal Utilities#

Internal utility APIs.

Microbatches Calculator
Optimizer Parameters Scheduler
- Weight decay on RMSNorm gamma

previous

dist_checkpointing.strategies package

next

Microbatches Calculator

Privacy Policy | Your Privacy Choices | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2026, NVIDIA Corporation.