Skip to main content

Ctrl+K

Megatron Core

GitHub

Megatron Core

GitHub

Table of Contents

Get Started

Quick Start

Basic Usage

Data Preparation
Training Examples
Parallelism Strategies Guide

Supported Models

Supported Models

Advanced Features

Mixture of Experts
- Multi-Token Prediction (MTP)
- Multi-Latent Attention
context_parallel package
Megatron FSDP
Distributed Optimizer
Optimizer CPU Offload
Custom Pipeline Model Parallel Layout
Megatron Energon
Megatron RL
Tokenizers

Developer Guide

Contributing to Megatron-LM
How to Submit a PR
Oncall Overview
Generating Docs Locally

Discussions

Discussions

API Reference

API Guide
API Reference
- core

API Reference
core
core.extensions

`core.extensions`#

Submodules#

core.extensions.transformer_engine_spec_provider
core.extensions.transformer_engine
core.extensions.kitchen

previous

core.models.backends

next

core.extensions.transformer_engine_spec_provider

On this page

Submodules

Privacy Policy | Your Privacy Choices | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2025, NVIDIA Corporation.