core.distributed.fsdp.src.megatron_fsdp.experimental

Experimental Megatron-FSDP implementation.

Submodules

• core.distributed.fsdp.src.megatron_fsdp.experimental.dbuffer
• core.distributed.fsdp.src.megatron_fsdp.experimental.uneven_dtensor
• core.distributed.fsdp.src.megatron_fsdp.experimental.optimizer
• core.distributed.fsdp.src.megatron_fsdp.experimental.module
• core.distributed.fsdp.src.megatron_fsdp.experimental.countdown
• core.distributed.fsdp.src.megatron_fsdp.experimental.quantized_dbuffer
• core.distributed.fsdp.src.megatron_fsdp.experimental.owner_planning
• core.distributed.fsdp.src.megatron_fsdp.experimental.parameter_group
• core.distributed.fsdp.src.megatron_fsdp.experimental.schedule
• core.distributed.fsdp.src.megatron_fsdp.experimental.module_utils
• core.distributed.fsdp.src.megatron_fsdp.experimental.placement
• core.distributed.fsdp.src.megatron_fsdp.experimental.checkpoint
• core.distributed.fsdp.src.megatron_fsdp.experimental.layout
• core.distributed.fsdp.src.megatron_fsdp.experimental.fully_shard
• core.distributed.fsdp.src.megatron_fsdp.experimental.indexed_order

Package Contents

Data

__all__

API

core.distributed.fsdp.src.megatron_fsdp.experimental.__all__

[‘DBuffer’, ‘Placements’, ‘SchedulePolicy’, ‘fully_shard’, ‘fully_shard_context’, ‘fully_shard_optim…