nemo_automodel.components.models.ernie5_5.checkpoint_layout
nemo_automodel.components.models.ernie5_5.checkpoint_layout
Name tables and eager conversions between the released ERNIE 5.5 checkpoint and the native modules.
Every conversion translates complete, unsharded tensors between the released Hugging Face layout and the native
module layout, one component at a time: the model-level tensors (token embedding, final norm, output head,
MultiMax and hyper-head), one hyper-connection (mHC) unit, one HCA attention layer, one DSA attention layer, one
MoE feed-forward, one decoder layer and the whole decoder-only model. The name tables are shared with the
distributed destinations and export views of checkpoint_io.
Memory. A conversion to the released layout returns views: the per-expert matrices and the hyper-head matrix are transposed views of the native storage, and the only tensor it allocates is the fused K/V projection of each DSA layer. A conversion to the native layout packs every routed expert of a layer into new grouped tensors.
Limitations:
- The conversions accept only complete, plain tensors.
map_model_from_hfholds a second copy of every routed expert, so it suits small checkpoints and shape-only tensors on the meta device, not the released checkpoint of several hundred GB. The conversions do not shard, stream from disk or export incrementally. - MTP weights are not mapped. The whole-model conversions raise for a config with MTP layers and reject any key outside the decoder layers and the model-level tensors.
- The decoder-layer conversions name the feed-forward
feed_forward.*for both kinds, while the model registers it asmlp.*(dense) ormoe.*(MoE);Ernie5_5StateDictAdapterrenames it.
Module Contents
Classes
Functions
Data
API
Per-head dimensions of the released fused K/V projection of a DSA layer.
Raises:
ValueError: If a dimension is not positive.
Number of attention heads.
Key rows per head in the fused matrix.
Latent width of the K/V projection (the columns of the fused matrix).
Value rows per head in the fused matrix.
Reject a layout with a dimension that is not positive.
Return the expert indices as a tuple after validating them.
Parameters:
Indices of routed experts.
Returns: tuple[int, ...]
The indices in the given order.
Raises:
ValueError: If the sequence is empty, an index is not a non-negative integer or an index repeats.
Return what the decoder-layer conversions need besides the kinds of one planned layer.
Parameters:
Model configuration; supplies the DSA K/V dimensions and the routed expert count.
Planned attention and feed-forward kinds of the layer.
Returns: DSAKVLayout | None
The K/V layout of a DSA layer (None for an HCA layer) and the indices of all routed experts in
Remove every entry whose name starts with prefix and return them without the prefix.
Parameters:
Names mapped to tensors of any shape. Matching entries are removed in place.
Name prefix to select and strip.
Returns: dict[str, torch.Tensor]
The removed entries keyed by the rest of their names, in their original order. Tensors are the
Add every entry of state to result under prefix + name.
Parameters:
Names mapped to tensors; updated in place.
Prefix prepended to every name of state.
Names mapped to tensors of any shape. Tensors are stored as given.
Require exactly the names of the eight model-level tensors.
Parameters:
Names mapped to tensors of any shape.
The eight names that must be present, with no others.
What the names are, for the message.
Raises:
ValueError: If a name is missing or unexpected. The message lists the missing names and the first five unexpected ones; the keys of an MTP block, which a decoder-only conversion does not map, end here.
Split a complete released fused K/V projection after checking its shape.
Parameters:
Tensor of shape [heads * (key_channels + value_channels), latent].
Per-head dimensions of the fused K/V projection.
Returns: torch.Tensor
Keys of shape [heads * latent, key_channels] and values of shape [heads * value_channels, latent], as
Raises:
ValueError: If the fused matrix does not have the layout’s shape.
Check the shapes of the five model-level tensors that keep their released layout.
Parameters:
Mapping that contains the Hugging Face names model.embed_tokens.weight [vocab, hidden],
lm_head.weight [vocab, hidden], model.norm.weight [hidden],
model.lm_head.multimax_ranges [4] and model.lm_head.multimax_ts [4]. Other entries are
ignored.
Vocabulary size.
Hidden size.
Raises:
ValueError: Ifvocaborhiddenis not positive or one of the tensors has another shape.
Require exactly the expected names of one mHC unit and FP32 tensors.
Parameters:
Relative mHC names mapped to tensors of any shape.
Names that must be present, with no others.
Raises:
ValueError: If a name is missing or unexpected, or a tensor is not FP32.
Check that the layer kinds are known and carry exactly the arguments they need.
Parameters:
"hca" or "dsa".
"dense" or "moe".
K/V layout; required for DSA and not accepted for HCA.
Routed experts; required for MoE and not accepted for a dense feed-forward.
Raises:
ValueError: If a kind is unknown or an argument is missing or superfluous.
Return the eight model-level tensors as native name -> (Hugging Face name, native shape).
The hyper-head matrix is the only one whose released layout differs from the native layout: the
checkpoint stores hc_head_fn as [streams * hidden, streams], the transpose of hc_fn.
Parameters:
Vocabulary size.
Hidden size.
Number of residual streams.
Returns: dict[str, tuple[str, tuple[int, ...]]]
Native names, relative to the inner model, mapped to the Hugging Face name and the native shape.
Return the per-head dimensions of the fused K/V projection of a configuration’s DSA layers.
Parameters:
Model configuration; supplies hybrid_mla_num_attention_heads,
hybrid_mla_kv_lora_rank, hybrid_mla_qk_nope_head_dim and hybrid_mla_v_head_dim.
Returns: DSAKVLayout
The head count, the latent width and the key and value rows per head.
Raises:
ValueError: If a dimension is not positive.
Split a released fused K/V projection and write the two parts into the native projections in place.
This completes a checkpoint read into an unsharded model: the fused projection is the one released tensor of a DSA layer that cannot be a view of model storage.
Parameters:
Tensor of shape [heads * (key_channels + value_channels), latent] in the released layout.
Native key projection of shape [heads * latent, key_channels]; overwritten in place, with a cast to its dtype.
Native value projection of shape [heads * value_channels, latent]; overwritten in place, with a cast to its dtype.
Per-head dimensions of the fused K/V projection.
Raises:
ValueError: If the fused matrix does not have the layout’s shape.
Map the eight model-level Hugging Face tensors to native names.
Parameters:
Exactly the eight released tensors model.embed_tokens.weight [vocab, hidden],
lm_head.weight [vocab, hidden], model.norm.weight [hidden],
model.lm_head.multimax_ranges [4], model.lm_head.multimax_ts [4],
model.mhc_contract.hc_head_fn [streams * hidden, streams],
model.mhc_contract.hc_head_base [streams] and model.mhc_contract.hc_head_scale [1].
The three hyper-head tensors must be FP32; the dtype of the others is not checked.
Vocabulary size.
Hidden size.
Number of residual streams.
Returns: dict[str, torch.Tensor]
Native tensors named relative to the inner model: embed_tokens.weight [vocab, hidden],
Raises:
ValueError: If the key set differs from the eight names, a dimension is not positive, a shape does not match or a hyper-head tensor is not FP32.
Restore the eight model-level Hugging Face tensors from their native names.
Parameters:
Exactly the eight native tensors, named relative to the inner model:
embed_tokens.weight [vocab, hidden], lm_head.weight [vocab, hidden], norm.weight
[hidden], multimax.multimax_ranges [4], multimax.multimax_ts [4], hyper_head.hc_fn
[streams, streams * hidden], hyper_head.hc_base [streams] and hyper_head.hc_scale [1].
The three hyper-head tensors must be FP32; the dtype of the others is not checked.
Vocabulary size.
Hidden size.
Number of residual streams.
Returns: dict[str, torch.Tensor]
The released tensors in the layouts listed for map_common_from_hf.
Raises:
ValueError: If the key set differs from the eight names, a dimension is not positive, a shape does not match or a hyper-head tensor is not FP32.
Map relative HF keys for one decoder layer to native block keys.
Selected experts are packed eagerly.
Parameters:
Complete tensors of one layer, named relative to model.layers.{index}:
input_layernorm.weight and post_attention_layernorm.weight (any shape, passed
through); self_attention_hyper_connection.* and mlp_hyper_connection.* in the layout
of map_mhc_from_hf; self_attn.* in the layout of map_hca_from_hf or
map_dsa_from_hf; and either mlp.gate_proj.weight, mlp.up_proj.weight and
mlp.down_proj.weight (any shape, passed through) or block_sparse_moe.* in the layout
of map_moe_from_hf.
Attention kind of the layer.
Feed-forward kind of the layer.
K/V layout; required for DSA and not accepted for HCA.
Routed experts to pack, in the order of the native expert axis; required for MoE and not accepted for a dense feed-forward.
Returns: dict[str, torch.Tensor]
Native tensors named relative to one decoder block: the two norm weights as given, attn_hc.*
Raises:
ValueError: If the kinds and their arguments are inconsistent, a key is missing or unexpected, or a component conversion rejects its tensors.
Restore one native decoder block to relative HF decoder-layer keys.
Parameters:
Complete tensors of one decoder block, named relative to it: input_layernorm.weight
and post_attention_layernorm.weight (any shape, passed through); attn_hc.* and
ffn_hc.* in the layout of map_mhc_to_hf; attention.* in the layout of
map_hca_to_hf or map_dsa_to_hf; and feed_forward.*, either the three dense
projection weights (any shape, passed through) or the layout of map_moe_to_hf.
Attention kind of the layer.
Feed-forward kind of the layer.
K/V layout; required for DSA and not accepted for HCA.
Routed experts, one per entry of the native expert axis and in its order; required for MoE and not accepted for a dense feed-forward.
Returns: dict[str, torch.Tensor]
Released tensors named relative to model.layers.{index}, in the layouts accepted by
Raises:
ValueError: If the kinds and their arguments are inconsistent, a key is missing or unexpected, or a component conversion rejects its tensors.
Convert all relative attention keys of one Hugging Face DSA layer to native keys.
Parameters:
Exactly the fifteen renamed suffixes relative to a single self_attn, mapped to tensors
of any shape and dtype (not checked, not copied), plus kv_b_proj.weight of shape
[heads * (key_channels + value_channels), latent].
Per-head dimensions of the fused K/V projection.
Returns: dict[str, torch.Tensor]
The renamed tensors as given, plus k_b_proj [heads * latent, key_channels] and v_b_proj
Raises:
ValueError: If a key is missing or unexpected, or the fused matrix does not have the layout’s shape.
Restore relative HF attention keys, retaining input tensor dtypes.
Parameters:
Exactly the fifteen renamed native names of one DSA attention, mapped to tensors of any
shape and dtype (not checked, not copied), plus k_b_proj [heads * latent, key_channels] and
v_b_proj [heads * value_channels, latent] with one common dtype.
Per-head dimensions of the fused K/V projection.
Returns: dict[str, torch.Tensor]
The renamed tensors as given, plus a new kv_b_proj.weight of shape
Raises:
ValueError: If a key is missing or unexpected, the K/V projections do not have the layout’s shapes or their dtypes differ.
Rename a complete single HCA layer’s relative keys without copying tensors.
Parameters:
Exactly the fifteen released suffixes relative to a single self_attn, mapped to tensors
of any shape and dtype. Shapes are not checked and no layout is changed.
Returns: dict[str, torch.Tensor]
The same tensor objects under the relative names expected by Ernie5_5HCAAttention.
Raises:
ValueError: If any expected key is missing or an unexpected key is present.
Restore a complete HCA component’s relative released key names.
Parameters:
Exactly the fifteen native names of one Ernie5_5HCAAttention, mapped to tensors of any
shape and dtype. Shapes are not checked and no layout is changed.
Returns: dict[str, torch.Tensor]
The same tensor objects under the released suffixes relative to a single self_attn.
Raises:
ValueError: If any expected key is missing or an unexpected key is present.
Map the released hyper-head tensors to the native stream-contraction module.
Parameters:
Exactly hc_head_fn [streams * hidden, streams], hc_head_base [streams] and
hc_head_scale [1], all FP32, named relative to model.mhc_contract.
Number of residual streams.
Hidden size.
Returns: dict[str, torch.Tensor]
hc_fn [streams, streams * hidden], the transpose of hc_head_fn made contiguous, and
Raises:
ValueError: If the key set differs, a dimension is not positive, a shape does not match or a tensor is not FP32.
Restore the released hyper-head tensors from the native stream-contraction module.
Parameters:
Exactly hc_fn [streams, streams * hidden], hc_base [streams] and hc_scale [1],
all FP32.
Number of residual streams.
Hidden size.
Returns: dict[str, torch.Tensor]
hc_head_fn [streams * hidden, streams], a transposed view of hc_fn, and hc_head_base
Raises:
ValueError: If the key set differs, a dimension is not positive, a shape does not match or a tensor is not FP32.
Convert the five released tensors of one mHC unit to the native layout.
The checkpoint stores the projection as [inputs, outputs] with three one-element alphas; the module
stores it as [outputs, inputs] with one scale vector. In the model inputs is streams * hidden
and outputs is (2 + streams) * streams; this function only checks that projection and bias agree.
Parameters:
Exactly mapping_proj.weight [inputs, outputs], bias [outputs], alpha_pre [1],
alpha_post [1] and alpha_res [1], all FP32, named relative to one hyper-connection.
Returns: dict[str, torch.Tensor]
fn [outputs, inputs], a transposed view of the projection; base [outputs], the input bias;
Raises:
ValueError: If the key set differs, a tensor is not FP32, the projection is not a matrix, the bias does not match the projection or an alpha does not have shape [1].
Convert one native mHC unit back to the five released tensors.
Parameters:
Exactly fn [outputs, inputs], base [outputs] and scale [3], all FP32.
Returns: dict[str, torch.Tensor]
mapping_proj.weight [inputs, outputs], a transposed view of fn; bias [outputs], the
Raises:
ValueError: If the key set differs, a tensor is not FP32,fnis not a matrix,basedoes not matchfnorscaledoes not have shape [3].
Map exactly a decoder-only HF state dict to the native names of the inner model.
All routed experts of every MoE layer are packed in memory, so this is meant for small checkpoints and for shape-only tensors on the meta device.
Parameters:
Complete, unsharded released tensors: model.layers.{index}.* for every decoder layer in
the layouts of map_decoder_from_hf, and the eight model-level tensors in the layouts of
map_common_from_hf.
Model configuration without MTP layers.
Returns: dict[str, torch.Tensor]
Native tensors named relative to the inner model: layers.{index}.* in the layouts returned by
Raises:
NotImplementedError: If the configuration has MTP layers.ValueError: If a key is missing or unexpected (MTP and unknown keys are rejected, not dropped) or a component conversion rejects its tensors.
Map exactly a decoder-only native state dict of the inner model back to HF names.
Parameters:
Complete, unsharded native tensors named relative to the inner model: layers.{index}.*
for every decoder layer in the layouts of map_decoder_to_hf, and the eight model-level
tensors in the layouts of map_common_to_hf.
Model configuration without MTP layers.
Returns: dict[str, torch.Tensor]
Released tensors: model.layers.{index}.* in the layouts returned by map_decoder_to_hf and
Raises:
NotImplementedError: If the configuration has MTP layers.ValueError: If a key is missing or unexpected or a component conversion rejects its tensors.
Map non-expert keys and selected experts of one HF MoE layer.
Parameters:
Exactly the ten released non-expert tensors of one block_sparse_moe (any shape and
dtype; not checked, not copied) and, for every selected expert, experts.{id}.w1.weight
[intermediate, latent], experts.{id}.w2.weight [latent, intermediate] and
experts.{id}.w3.weight [intermediate, latent].
Unique non-negative expert indices, in the order of the native expert axis.
Returns: dict[str, torch.Tensor]
The non-expert tensors as given under their native names, plus experts.gate_and_up_projs
Raises:
ValueError: If the indices are invalid, a key is missing or unexpected, or the experts cannot be packed.
Restore a local MoE’s grouped experts and non-expert keys to HF names.
Parameters:
Exactly the ten native non-expert tensors of one MoE (any shape and dtype; not checked,
not copied), experts.gate_and_up_projs [experts, latent, 2 * intermediate] and
experts.down_projs [experts, intermediate, latent].
Unique non-negative expert indices, one per entry of the expert axis and in its order.
Returns: dict[str, torch.Tensor]
The non-expert tensors as given under their released names, plus the three per-expert matrices
Raises:
ValueError: If the indices are invalid, a key is missing or unexpected, or the grouped tensors do not match the indices.
Pack selected HF experts in the supplied order into grouped matrices.
Only the selected experts are allocated.
Parameters:
Mapping that contains, for every selected expert, experts.{id}.w1.weight (gate
projection) [intermediate, latent], experts.{id}.w2.weight (down projection)
[latent, intermediate] and experts.{id}.w3.weight (up projection) [intermediate, latent].
The selected tensors must share one dtype and one device. Other entries are ignored.
Unique non-negative expert indices. Their order is the order along the expert axis of the results.
Returns: torch.Tensor
Two new tensors with the dtype and device of the inputs: gate/up of shape
Raises:
ValueError: If the indices are invalid, a selected key is missing, the first gate projection is not a matrix, the shapes of an expert do not match, or dtypes or devices differ.
Split the released fused K/V projection into the separate native projections.
The fused matrix stores, for each head in turn, key_channels key rows followed by
value_channels value rows, every row of length latent.
Parameters:
Tensor of shape [heads * (key_channels + value_channels), latent]. It may be the global
matrix or one rank’s shard of whole heads, with heads counting the heads it holds.
Number of attention heads held by combined.
Latent width of the K/V projection.
Key rows per head.
Value rows per head.
Returns: torch.Tensor
Keys of shape [heads * latent, key_channels], each head’s key block transposed, and values of
Restore selected grouped experts to their HF per-expert matrices.
Parameters:
Tensor of shape [experts, latent, 2 * intermediate]. The last axis holds the transposed gate projection followed by the transposed up projection.
Tensor of shape [experts, intermediate, latent] with the dtype and device of gate_up.
Unique non-negative expert indices, one per entry of the expert axis and in its order.
Returns: dict[str, torch.Tensor]
For every index, experts.{id}.w1.weight [intermediate, latent], experts.{id}.w2.weight
Raises:
ValueError: If the indices are invalid,gate_updoes not have three axes with an even last axis and one entry per index, ordowndoes not match it in shape, dtype or device.