core.distributed

Subpackages

• core.distributed.fsdp
  • core.distributed.fsdp.src
    • core.distributed.fsdp.src.megatron_fsdp
  • core.distributed.fsdp.mcore_fsdp_adapter

Submodules

• core.distributed.param_and_grad_buffer
• core.distributed.torch_fully_sharded_data_parallel
• core.distributed.torch_fully_sharded_data_parallel_config
• core.distributed.finalize_model_grads
• core.distributed.distributed_data_parallel_config
• core.distributed.reduce_scatter_with_fp32_accumulation
• core.distributed.data_parallel_base
• core.distributed.distributed_data_parallel