nemo_automodel.components.models.deepseek_v4.parallelization

Model-owned distributed parallelization for DeepSeek-V4.

Module Contents

Name	Description
`DeepseekV4ModelParallelizer`	Keep DeepSeek-V4 reference-sensitive parameters in fp32 FSDP units.

class nemo_automodel.components.models.deepseek_v4.parallelization.DeepseekV4ModelParallelizer()

Bases: ModelParallelizer

Keep DeepSeek-V4 reference-sensitive parameters in fp32 FSDP units.

nemo_automodel.components.models.deepseek_v4.parallelization.DeepseekV4ModelParallelizer._fully_shard_module(
    module,
    kwargs = {}
)

nemo_automodel.components.models.deepseek_v4.parallelization.PARALLELIZER = DeepseekV4ModelParallelizer()

nemo_automodel.components.models.deepseek_v4.parallelization.__all__ = ['PARALLELIZER']