nemo_automodel.components.distributed

Subpackages

• nemo_automodel.components.distributed.pipelining
  • nemo_automodel.components.distributed.pipelining.autopipeline
  • nemo_automodel.components.distributed.pipelining.config
  • nemo_automodel.components.distributed.pipelining.functional
  • nemo_automodel.components.distributed.pipelining.hf_utils

Submodules

• nemo_automodel.components.distributed.parallelizer_utils
• nemo_automodel.components.distributed.cp_utils
• nemo_automodel.components.distributed.thd_utils
• nemo_automodel.components.distributed.grad_utils
• nemo_automodel.components.distributed.utils
• nemo_automodel.components.distributed.ddp
• nemo_automodel.components.distributed.megatron_fsdp
• nemo_automodel.components.distributed.parallelizer
• nemo_automodel.components.distributed.config
• nemo_automodel.components.distributed.parallel_styles
• nemo_automodel.components.distributed.init_utils
• nemo_automodel.components.distributed.mesh
• nemo_automodel.components.distributed.fsdp2
• nemo_automodel.components.distributed.optimized_tp_plans
• nemo_automodel.components.distributed.mesh_utils
• nemo_automodel.components.distributed.tensor_utils

Package Contents

Data

__all__

API

nemo_automodel.components.distributed.__all__

[‘FSDP2Config’, ‘MegatronFSDPConfig’, ‘DDPConfig’, ‘PipelineConfig’]