API Guide# API reference documentation for Megatron Core components. Model APIs models package Subpackages models.gpt package models.bert package models.t5 package Core APIs transformer package tensor_parallel package Generalized Tensor Parallelism (GTP) 1. Features 2. Usage 3. Implementation details 4. Testing pipeline_parallel package fusions package distributed package datasets package Data Pipeline Data pre-processing Data loading: construction Data loading: implementation Offline cache preparation Fast DataLoader initialization dist_checkpointing package Safe Checkpoint Loading Checkpointing Distributed Optimizer Distributed Optimizer Checkpoint Formats Async Checkpoint Saving Strategy Subpackages dist_checkpointing.strategies package Internal Utilities Microbatches Calculator Optimizer Parameters Scheduler
