Skip to navigation

nemo_automodel.components.models.qwen3_8_flash_next.parallelization

View as MarkdownOpen in Claude

Distributed-parallelization registration for Qwen3.8-Flash-Next.

Module Contents

Classes

NameDescription
Qwen3_8_FlashNextModelParallelizerReject dense FSDP2 for PLE checkpoints, which never binds the PLE owner group.

Data

PARALLELIZER

API

class nemo_automodel.components.models.qwen3_8_flash_next.parallelization.Qwen3_8_FlashNextModelParallelizer()

Bases: ModelParallelizer

Reject dense FSDP2 for PLE checkpoints, which never binds the PLE owner group.

nemo_automodel.components.models.qwen3_8_flash_next.parallelization.Qwen3_8_FlashNextModelParallelizer._apply(
model: torch.nn.Module,
device_mesh: torch.distributed.device_mesh.DeviceMesh,
kwargs = {}
) -> torch.nn.Module

Apply dense FSDP2, which the shared dispatch selects only for ep_size == 1.

Only the MoE parallelizer (ep_size > 1) binds the PLE table’s owner group to the stage’s data/context shard mesh. The check reads the config rather than the stage’s modules, so every pipeline stage raises together instead of only the stage that holds the table.

Parameters:

model
nn.Module

Qwen3.8-Flash-Next model or pipeline stage.

device_mesh
DeviceMesh

Device mesh of the dense FSDP2 path.

**kwargs
Defaults to {}

Remaining arguments of ModelParallelizer._apply.

Returns: nn.Module

The parallelized model.

nemo_automodel.components.models.qwen3_8_flash_next.parallelization.PARALLELIZER = Qwen3_8_FlashNextModelParallelizer()