nemo_automodel.components.models.qwen3_moe.state_dict_adapter

Module Contents

Classes

Name	Description
`Qwen3MoeStateDictAdapter`	Converts between HF Qwen3-MoE checkpoints and our grouped-experts native format.

Data

_LORA_EXPERT_SUFFIXES

logger

API

class nemo_automodel.components.models.qwen3_moe.state_dict_adapter.Qwen3MoeStateDictAdapter(
    config: typing.Any,
    moe_config: nemo_automodel.components.moe.config.MoEConfig,
    backend: nemo_automodel.components.models.common.BackendConfig,
    dtype: torch.dtype = torch.float32
)

Bases: MoESplitExpertsStateDictMixin, StateDictAdapter

Converts between HF Qwen3-MoE checkpoints and our grouped-experts native format.

_v5_peft_target_parameters

tuple[str, ...]

Qwen3 MoE is validated for fused PEFT v5 ParamWrapper export.

nemo_automodel.components.models.qwen3_moe.state_dict_adapter.Qwen3MoeStateDictAdapter.convert_single_tensor_to_hf(
    fqn: str,
    tensor: typing.Any,
    kwargs = {}
) -> list[tuple[str, typing.Any]]

Convert a single tensor from native format to HuggingFace format.

When v4_compatible=False (the default), LoRA expert tensors are emitted in PEFT v0.18+ ParamWrapper format so that PeftModel.from_pretrained() can load them directly. When v4_compatible=True, the legacy per-expert split is used instead (via the parent mixin).

Parameters:

fqn

str

Fully qualified name of the tensor in native format

tensor

Any

The tensor to convert

**kwargs

Defaults to {}

Additional arguments for conversion

Returns: list[tuple[str, Any]]

List of (fqn, tensor) tuples in HuggingFace format

nemo_automodel.components.models.qwen3_moe.state_dict_adapter.Qwen3MoeStateDictAdapter.from_hf(
    hf_state_dict: dict[str, typing.Any],
    device_mesh: typing.Optional[torch.distributed.device_mesh.DeviceMesh] = None,
    kwargs = {}
) -> dict[str, typing.Any]

Convert HF checkpoint to native format, handling ParamWrapper LoRA keys.

Before delegating to the parent _from_hf_w_merged_experts (which handles legacy per-expert LoRA format), this method scans for ParamWrapper-format LoRA keys and converts them back to the native grouped format expected by GroupedExpertsLoRA.

nemo_automodel.components.models.qwen3_moe.state_dict_adapter.Qwen3MoeStateDictAdapter.to_hf(
    state_dict: dict[str, typing.Any],
    exclude_key_regex: typing.Optional[str] = None,
    quantization: bool = False,
    kwargs = {}
) -> dict[str, typing.Any]

nemo_automodel.components.models.qwen3_moe.state_dict_adapter._LORA_EXPERT_SUFFIXES = ('lora_gate_and_up_A', 'lora_gate_and_up_B', 'lora_down_A', 'lora_down_B')

nemo_automodel.components.models.qwen3_moe.state_dict_adapter.logger = logging.getLogger(__name__)