nemo_automodel.components.models.beam.state_dict_adapter
nemo_automodel.components.models.beam.state_dict_adapter
State-dict adapter between Hugging Face Beam checkpoints and the native grouped-experts layout.
Module Contents
Classes
Data
API
Bases: Qwen3MoeStateDictAdapter
Converts between Hugging Face Beam checkpoints and the grouped-experts native format.
Beam stores its experts exactly like Qwen3-MoE, so the Qwen3-MoE conversion (including fused PEFT v5
ParamWrapper LoRA export/import and low-memory DCP loading) applies unchanged. The native module tree
mirrors the HF one, so every tensor outside the experts keeps its HF name and passes through unchanged
(attention projections and gate_bias, q/k norms, the four per-layer norms, the dense MLP,
mlp.gate.weight and the mlp.gate.e_score_correction_bias buffer, embeddings, final norm,
lm_head). Experts differ:
HF checkpoint (per expert, the qwen2_moe conversion mapping)::
model.layers.{L}.mlp.experts.{E}.gate_proj.weight [moe_inter_dim, hidden] model.layers.{L}.mlp.experts.{E}.up_proj.weight [moe_inter_dim, hidden] model.layers.{L}.mlp.experts.{E}.down_proj.weight [hidden, moe_inter_dim]
Native (grouped)::
model.layers.{L}.mlp.experts.gate_and_up_projs [n_experts, hidden, 2 * moe_inter_dim] model.layers.{L}.mlp.experts.down_projs [n_experts, moe_inter_dim, hidden]
to_hf therefore writes the per-expert layout that transformers itself produces with
save_pretrained, not the stacked in-memory layout.
With set_expert_storage_format("nvfp4") (selected by apply_nvfp4_to_moe_experts for experts built
on meta) the routed experts are instead mapped onto the packed parameters of NVFP4ExpertStorageMixin
without ever materializing them in floating point. A ModelOpt NVFP4 export stores each expert projection
as weight (uint8 fp4-e2m1 codes, two per byte, [out, in / 2]), weight_scale (float8_e4m3fn,
one per 16 columns, [out, in / 16]) and weight_scale_2 (float32 per-tensor scale, 0-dim); the
inference-only input_scale is dropped. Natively, gate and up rows are concatenated (gate first) and the
experts stacked::
model.layers.{L}.mlp.experts.gate_and_up_projs_nvfp4 uint8 [n_experts, 2 * moe_inter_dim, hidden / 2] model.layers.{L}.mlp.experts.gate_and_up_projs_scales float8_e4m3fn [n_experts, 2 * moe_inter_dim, hidden / 16] model.layers.{L}.mlp.experts.gate_and_up_projs_global float32 [n_experts, 2 * moe_inter_dim] model.layers.{L}.mlp.experts.down_projs_nvfp4 uint8 [n_experts, hidden, moe_inter_dim / 2] model.layers.{L}.mlp.experts.down_projs_scales float8_e4m3fn [n_experts, hidden, moe_inter_dim / 16] model.layers.{L}.mlp.experts.down_projs_global float32 [n_experts, hidden]
The per-tensor scale is kept once per output row (*_global) so the fused gate/up parameter carries the
two projections’ scales exactly. During a checkpoint load the codes and block scales are exposed as
per-expert views into the model’s own storage (DCP writes the checkpoint bytes straight through them,
including the EP-sharded path), and the 0-dim weight_scale_2 entries are read into scalars that
from_hf expands per row.
Per-projection checkpoint shape [out, in] of the routed experts (before packing).
Pop the per-expert NVFP4 tensors of this rank’s experts and stack them into the packed parameters.
Parameters:
Checkpoint mapping; every routed-expert weight / weight_scale /
weight_scale_2 / input_scale entry is removed from it.
Optional expert-parallel mesh selecting the local experts.
Returns: dict[str, Any]
The packed native entries (*_nvfp4 uint8, *_scales float8_e4m3fn, *_global
Raises:
RuntimeError: If a required expert tensor is missing.ValueError: If a tensor does not have the ModelOpt NVFP4 dtype or shape.
Split one grouped packed expert parameter into per-expert ModelOpt checkpoint tensors.
Parameters:
Native parameter name; returns None unless it is a packed expert parameter.
Grouped tensor [experts, ...] in checkpoint orientation (gate rows before up rows for the
fused projection). An EP DTensor contributes only the experts local to this rank.
Emit DCP destinations: the codes and block scales are views into tensor
(registered as loaded in place when it is model storage, i.e. a DTensor or a CUDA tensor) and
each weight_scale_2 is a fresh 0-dim tensor that from_hf expands per row.
Other export options, ignored here.
Returns: list[tuple[str, Any]] | None
(hf_key, tensor) pairs: weight uint8 [out, in / 2], weight_scale float8_e4m3fn
Validate one ModelOpt NVFP4 checkpoint tensor and return it in the native per-expert layout.
Returns: torch.Tensor
weight as uint8 [out, in / 2], weight_scale as float8_e4m3fn [out, in / 16] and
Collapse one projection’s per-row scales [out] to the 0-dim per-tensor scale ModelOpt stores.
Shape-only (meta) tensors, as passed by get_hf_state_dict_keys before sharding, skip the value check.
Convert one native tensor to the HF layout, splitting packed NVFP4 experts per expert.
Packed expert parameters (*_nvfp4 / *_scales / *_global) only exist on NVFP4-resident
modules, so they are mapped regardless of the selected storage format; everything else follows the
Qwen3-MoE conversion.
Convert an HF checkpoint to the native format.
With NVFP4 expert storage the per-expert packed tensors are aggregated first (see
_aggregate_nvfp4_experts); every other key, including LoRA adapters, follows the Qwen3-MoE path.
Select the routed-expert storage emitted during checkpoint loading ("bf16" or "nvfp4").