core.distributed.fsdp.src.megatron_fsdp

Subpackages

• core.distributed.fsdp.src.megatron_fsdp.experimental
  • core.distributed.fsdp.src.megatron_fsdp.experimental.parameter_group
  • core.distributed.fsdp.src.megatron_fsdp.experimental.optimizer
  • core.distributed.fsdp.src.megatron_fsdp.experimental.fully_shard
  • core.distributed.fsdp.src.megatron_fsdp.experimental.module
  • core.distributed.fsdp.src.megatron_fsdp.experimental.dbuffer
  • core.distributed.fsdp.src.megatron_fsdp.experimental.layout
  • core.distributed.fsdp.src.megatron_fsdp.experimental.placement
  • core.distributed.fsdp.src.megatron_fsdp.experimental.indexed_order

Submodules

• core.distributed.fsdp.src.megatron_fsdp.param_and_grad_buffer
• core.distributed.fsdp.src.megatron_fsdp.distributed_data_parallel_config
• core.distributed.fsdp.src.megatron_fsdp.fully_shard
• core.distributed.fsdp.src.megatron_fsdp.mixed_precision
• core.distributed.fsdp.src.megatron_fsdp.megatron_fsdp
• core.distributed.fsdp.src.megatron_fsdp.uneven_dtensor
• core.distributed.fsdp.src.megatron_fsdp.utils
• core.distributed.fsdp.src.megatron_fsdp.package_info

Package Contents

Data

__all__

API

core.distributed.fsdp.src.megatron_fsdp.__all__

[‘DistributedDataParallelConfig’, ‘MegatronFSDP’, ‘FSDPDistributedIndex’, ‘MixedPrecisionPolicy’, ‘f…