Skip to navigation
ReferenceFull Library ReferenceNemo AutomodelNemo AutomodelComponentsModelsBeamnemo_automodel.components.models.beam.state_dict_adapter

nemo_automodel.components.models.beam.state_dict_adapter

View as Markdown

State-dict adapter between Hugging Face Beam checkpoints and the native grouped-experts layout.

Module Contents

Classes

NameDescription
BeamStateDictAdapterConverts between Hugging Face Beam checkpoints and the grouped-experts native format.

Data

_NVFP4_CHECKPOINT_SUFFIX

_NVFP4_PARAM_SUFFIX

_NVFP4_PROJECTIONS

API

class nemo_automodel.components.models.beam.state_dict_adapter.BeamStateDictAdapter(
config: transformers.models.beam.BeamConfig,
dtype: torch.dtype = torch.float32,
expert_storage_format: str = 'bf16'
)

Bases: Qwen3MoeStateDictAdapter

Converts between Hugging Face Beam checkpoints and the grouped-experts native format.

Beam stores its experts exactly like Qwen3-MoE, so the Qwen3-MoE conversion (including fused PEFT v5 ParamWrapper LoRA export/import and low-memory DCP loading) applies unchanged. The native module tree mirrors the HF one, so every tensor outside the experts keeps its HF name and passes through unchanged (attention projections and gate_bias, q/k norms, the four per-layer norms, the dense MLP, mlp.gate.weight and the mlp.gate.e_score_correction_bias buffer, embeddings, final norm, lm_head). Experts differ:

HF checkpoint (per expert, the qwen2_moe conversion mapping)::

model.layers.{L}.mlp.experts.{E}.gate_proj.weight [moe_inter_dim, hidden] model.layers.{L}.mlp.experts.{E}.up_proj.weight [moe_inter_dim, hidden] model.layers.{L}.mlp.experts.{E}.down_proj.weight [hidden, moe_inter_dim]

Native (grouped)::

model.layers.{L}.mlp.experts.gate_and_up_projs [n_experts, hidden, 2 * moe_inter_dim] model.layers.{L}.mlp.experts.down_projs [n_experts, moe_inter_dim, hidden]

to_hf therefore writes the per-expert layout that transformers itself produces with save_pretrained, not the stacked in-memory layout.

With set_expert_storage_format("nvfp4") (selected by apply_nvfp4_to_moe_experts for experts built on meta) the routed experts are instead mapped onto the packed parameters of NVFP4ExpertStorageMixin without ever materializing them in floating point. A ModelOpt NVFP4 export stores each expert projection as weight (uint8 fp4-e2m1 codes, two per byte, [out, in / 2]), weight_scale (float8_e4m3fn, one per 16 columns, [out, in / 16]) and weight_scale_2 (float32 per-tensor scale, 0-dim); the inference-only input_scale is dropped. Natively, gate and up rows are concatenated (gate first) and the experts stacked::

model.layers.{L}.mlp.experts.gate_and_up_projs_nvfp4 uint8 [n_experts, 2 * moe_inter_dim, hidden / 2] model.layers.{L}.mlp.experts.gate_and_up_projs_scales float8_e4m3fn [n_experts, 2 * moe_inter_dim, hidden / 16] model.layers.{L}.mlp.experts.gate_and_up_projs_global float32 [n_experts, 2 * moe_inter_dim] model.layers.{L}.mlp.experts.down_projs_nvfp4 uint8 [n_experts, hidden, moe_inter_dim / 2] model.layers.{L}.mlp.experts.down_projs_scales float8_e4m3fn [n_experts, hidden, moe_inter_dim / 16] model.layers.{L}.mlp.experts.down_projs_global float32 [n_experts, hidden]

The per-tensor scale is kept once per output row (*_global) so the fused gate/up parameter carries the two projections’ scales exactly. During a checkpoint load the codes and block scales are exposed as per-expert views into the model’s own storage (DCP writes the checkpoint bytes straight through them, including the EP-sharded path), and the 0-dim weight_scale_2 entries are read into scalars that from_hf expands per row.

_nvfp4_expert_dims
dict[str, tuple[int, int]]

Per-projection checkpoint shape [out, in] of the routed experts (before packing).

expert_storage_format
= 'bf16'
nemo_automodel.components.models.beam.state_dict_adapter.BeamStateDictAdapter._aggregate_nvfp4_experts(
hf_state_dict: dict[str, typing.Any],
device_mesh: torch.distributed.device_mesh.DeviceMesh | None
) -> tuple[dict[str, typing.Any], set[str]]

Pop the per-expert NVFP4 tensors of this rank’s experts and stack them into the packed parameters.

Parameters:

hf_state_dict
dict[str, Any]

Checkpoint mapping; every routed-expert weight / weight_scale / weight_scale_2 / input_scale entry is removed from it.

device_mesh
DeviceMesh | None

Optional expert-parallel mesh selecting the local experts.

Returns: dict[str, Any]

The packed native entries (*_nvfp4 uint8, *_scales float8_e4m3fn, *_global

Raises:

  • RuntimeError: If a required expert tensor is missing.
  • ValueError: If a tensor does not have the ModelOpt NVFP4 dtype or shape.
nemo_automodel.components.models.beam.state_dict_adapter.BeamStateDictAdapter._convert_nvfp4_expert_to_hf(
fqn: str,
tensor: typing.Any,
for_checkpoint_load: bool = False,
kwargs = {}
) -> list[tuple[str, typing.Any]] | None

Split one grouped packed expert parameter into per-expert ModelOpt checkpoint tensors.

Parameters:

fqn
str

Native parameter name; returns None unless it is a packed expert parameter.

tensor
Any

Grouped tensor [experts, ...] in checkpoint orientation (gate rows before up rows for the fused projection). An EP DTensor contributes only the experts local to this rank.

for_checkpoint_load
boolDefaults to False

Emit DCP destinations: the codes and block scales are views into tensor (registered as loaded in place when it is model storage, i.e. a DTensor or a CUDA tensor) and each weight_scale_2 is a fresh 0-dim tensor that from_hf expands per row.

**kwargs
Defaults to {}

Other export options, ignored here.

Returns: list[tuple[str, Any]] | None

(hf_key, tensor) pairs: weight uint8 [out, in / 2], weight_scale float8_e4m3fn

nemo_automodel.components.models.beam.state_dict_adapter.BeamStateDictAdapter._nvfp4_checkpoint_tensor(
key: str,
projection: str,
suffix: str,
value: torch.Tensor
) -> torch.Tensor

Validate one ModelOpt NVFP4 checkpoint tensor and return it in the native per-expert layout.

Returns: torch.Tensor

weight as uint8 [out, in / 2], weight_scale as float8_e4m3fn [out, in / 16] and

nemo_automodel.components.models.beam.state_dict_adapter.BeamStateDictAdapter._nvfp4_tensor_scale(
row_scales: torch.Tensor,
key: str
) -> torch.Tensor
staticmethod

Collapse one projection’s per-row scales [out] to the 0-dim per-tensor scale ModelOpt stores.

Shape-only (meta) tensors, as passed by get_hf_state_dict_keys before sharding, skip the value check.

nemo_automodel.components.models.beam.state_dict_adapter.BeamStateDictAdapter.convert_single_tensor_to_hf(
fqn: str,
tensor: typing.Any,
kwargs = {}
) -> list[tuple[str, typing.Any]]

Convert one native tensor to the HF layout, splitting packed NVFP4 experts per expert.

Packed expert parameters (*_nvfp4 / *_scales / *_global) only exist on NVFP4-resident modules, so they are mapped regardless of the selected storage format; everything else follows the Qwen3-MoE conversion.

nemo_automodel.components.models.beam.state_dict_adapter.BeamStateDictAdapter.from_hf(
hf_state_dict: dict[str, typing.Any],
device_mesh: torch.distributed.device_mesh.DeviceMesh | None = None,
kwargs = {}
) -> dict[str, typing.Any]

Convert an HF checkpoint to the native format.

With NVFP4 expert storage the per-expert packed tensors are aggregated first (see _aggregate_nvfp4_experts); every other key, including LoRA adapters, follows the Qwen3-MoE path.

nemo_automodel.components.models.beam.state_dict_adapter.BeamStateDictAdapter.set_expert_storage_format(
storage_format: str
) -> None

Select the routed-expert storage emitted during checkpoint loading ("bf16" or "nvfp4").

nemo_automodel.components.models.beam.state_dict_adapter._NVFP4_CHECKPOINT_SUFFIX = {'_nvfp4': 'weight', '_scales': 'weight_scale', '_global': 'weight_scale_2'}
nemo_automodel.components.models.beam.state_dict_adapter._NVFP4_PARAM_SUFFIX = {ckpt: param for param, ckpt in (_NVFP4_CHECKPOINT_SUFFIX.items())}
nemo_automodel.components.models.beam.state_dict_adapter._NVFP4_PROJECTIONS = {'gate_and_up_projs': ('gate_proj', 'up_proj'), 'down_projs': ('down_proj',)}