core.distributed.fsdp.src.megatron_fsdp.experimental

Experimental Megatron-FSDP implementation.

Submodules

• core.distributed.fsdp.src.megatron_fsdp.experimental.module
• core.distributed.fsdp.src.megatron_fsdp.experimental.uneven_dtensor
• core.distributed.fsdp.src.megatron_fsdp.experimental.checkpoint
• core.distributed.fsdp.src.megatron_fsdp.experimental.layout_builder
• core.distributed.fsdp.src.megatron_fsdp.experimental.countdown
• core.distributed.fsdp.src.megatron_fsdp.experimental.fully_shard
• core.distributed.fsdp.src.megatron_fsdp.experimental.optimizer
• core.distributed.fsdp.src.megatron_fsdp.experimental.dbuffer
• core.distributed.fsdp.src.megatron_fsdp.experimental.runtime_policy
• core.distributed.fsdp.src.megatron_fsdp.experimental.execution_order
• core.distributed.fsdp.src.megatron_fsdp.experimental.phase
• core.distributed.fsdp.src.megatron_fsdp.experimental.schedule_policy
• core.distributed.fsdp.src.megatron_fsdp.experimental.muon
• core.distributed.fsdp.src.megatron_fsdp.experimental.placement
• core.distributed.fsdp.src.megatron_fsdp.experimental.owner_planning
• core.distributed.fsdp.src.megatron_fsdp.experimental.layout
• core.distributed.fsdp.src.megatron_fsdp.experimental.parameter_group
• core.distributed.fsdp.src.megatron_fsdp.experimental.gather_scatter
• core.distributed.fsdp.src.megatron_fsdp.experimental.module_utils
• core.distributed.fsdp.src.megatron_fsdp.experimental.quantized_dbuffer
• core.distributed.fsdp.src.megatron_fsdp.experimental.range
• core.distributed.fsdp.src.megatron_fsdp.experimental.context

Package Contents

Data

__all__

API

core.distributed.fsdp.src.megatron_fsdp.experimental.__all__

[‘DBuffer’, ‘FsdpContext’, ‘Placements’, ‘PrefetchOrder’, ‘RuntimePolicy’, ‘SchedulePolicy’, ‘ShardL…