nemo_automodel.components.models.qwen3_8_flash_next.parallelization
nemo_automodel.components.models.qwen3_8_flash_next.parallelization
Distributed-parallelization registration for Qwen3.8-Flash-Next.
Module Contents
Classes
Data
API
Bases: ModelParallelizer
Reject dense FSDP2 for PLE checkpoints, which never binds the PLE owner group.
Apply dense FSDP2, which the shared dispatch selects only for ep_size == 1.
Only the MoE parallelizer (ep_size > 1) binds the PLE table’s owner
group to the stage’s data/context shard mesh. The check reads the
config rather than the stage’s modules, so every pipeline stage raises
together instead of only the stage that holds the table.
Parameters:
Qwen3.8-Flash-Next model or pipeline stage.
Device mesh of the dense FSDP2 path.
Remaining arguments of ModelParallelizer._apply.
Returns: nn.Module
The parallelized model.