Skip to navigation
ReferenceFull Library ReferenceNemo AutomodelNemo AutomodelComponentsModelsErnie5 5nemo_automodel.components.models.ernie5_5.checkpoint_layout

nemo_automodel.components.models.ernie5_5.checkpoint_layout

View as MarkdownOpen in Claude

Name tables and eager conversions between the released ERNIE 5.5 checkpoint and the native modules.

Every conversion translates complete, unsharded tensors between the released Hugging Face layout and the native module layout, one component at a time: the model-level tensors (token embedding, final norm, output head, MultiMax and hyper-head), one hyper-connection (mHC) unit, one HCA attention layer, one DSA attention layer, one MoE feed-forward, one decoder layer and the whole decoder-only model. The name tables are shared with the distributed destinations and export views of checkpoint_io.

Memory. A conversion to the released layout returns views: the per-expert matrices and the hyper-head matrix are transposed views of the native storage, and the only tensor it allocates is the fused K/V projection of each DSA layer. A conversion to the native layout packs every routed expert of a layer into new grouped tensors.

Limitations:

  • The conversions accept only complete, plain tensors. map_model_from_hf holds a second copy of every routed expert, so it suits small checkpoints and shape-only tensors on the meta device, not the released checkpoint of several hundred GB. The conversions do not shard, stream from disk or export incrementally.
  • MTP weights are not mapped. The whole-model conversions raise for a config with MTP layers and reject any key outside the decoder layers and the model-level tensors.
  • The decoder-layer conversions name the feed-forward feed_forward.* for both kinds, while the model registers it as mlp.* (dense) or moe.* (MoE); Ernie5_5StateDictAdapter renames it.

Module Contents

Classes

NameDescription
DSAKVLayoutPer-head dimensions of the released fused K/V projection of a DSA layer.

Functions

NameDescription
_checked_idsReturn the expert indices as a tuple after validating them.
_layer_layout_and_expertsReturn what the decoder-layer conversions need besides the kinds of one planned layer.
_pop_prefixRemove every entry whose name starts with prefix and return them without the prefix.
_put_prefixAdd every entry of state to result under prefix + name.
_require_common_keysRequire exactly the names of the eight model-level tensors.
_split_fused_kv_ofSplit a complete released fused K/V projection after checking its shape.
_validate_directCheck the shapes of the five model-level tensors that keep their released layout.
_validate_keysRequire exactly the expected names of one mHC unit and FP32 tensors.
_validate_kindsCheck that the layer kinds are known and carry exactly the arguments they need.
common_tensor_specsReturn the eight model-level tensors as native name -> (Hugging Face name, native shape).
dsa_kv_layoutReturn the per-head dimensions of the fused K/V projection of a configuration’s DSA layers.
install_fused_kvSplit a released fused K/V projection and write the two parts into the native projections in place.
map_common_from_hfMap the eight model-level Hugging Face tensors to native names.
map_common_to_hfRestore the eight model-level Hugging Face tensors from their native names.
map_decoder_from_hfMap relative HF keys for one decoder layer to native block keys.
map_decoder_to_hfRestore one native decoder block to relative HF decoder-layer keys.
map_dsa_from_hfConvert all relative attention keys of one Hugging Face DSA layer to native keys.
map_dsa_to_hfRestore relative HF attention keys, retaining input tensor dtypes.
map_hca_from_hfRename a complete single HCA layer’s relative keys without copying tensors.
map_hca_to_hfRestore a complete HCA component’s relative released key names.
map_hyper_head_from_hfMap the released hyper-head tensors to the native stream-contraction module.
map_hyper_head_to_hfRestore the released hyper-head tensors from the native stream-contraction module.
map_mhc_from_hfConvert the five released tensors of one mHC unit to the native layout.
map_mhc_to_hfConvert one native mHC unit back to the five released tensors.
map_model_from_hfMap exactly a decoder-only HF state dict to the native names of the inner model.
map_model_to_hfMap exactly a decoder-only native state dict of the inner model back to HF names.
map_moe_from_hfMap non-expert keys and selected experts of one HF MoE layer.
map_moe_to_hfRestore a local MoE’s grouped experts and non-expert keys to HF names.
pack_expertsPack selected HF experts in the supplied order into grouped matrices.
split_fused_kvSplit the released fused K/V projection into the separate native projections.
unpack_expertsRestore selected grouped experts to their HF per-expert matrices.

Data

COMMON_KEYS

DSA_HF_TO_NATIVE

HCA_HF_TO_NATIVE

MOE_HF_TO_NATIVE

_COMMON_NAMES

_DENSE

_DIRECT

_HEAD_KEYS

_HEAD_PREFIX

_HF_KEYS

_NATIVE_KEYS

_NORMS

_NORM_KEYS

API

class nemo_automodel.components.models.ernie5_5.checkpoint_layout.DSAKVLayout(
heads: int,
latent: int,
key_channels: int,
value_channels: int
)
Dataclass

Per-head dimensions of the released fused K/V projection of a DSA layer.

Raises:

  • ValueError: If a dimension is not positive.
heads
int

Number of attention heads.

key_channels
int

Key rows per head in the fused matrix.

latent
int

Latent width of the K/V projection (the columns of the fused matrix).

value_channels
int

Value rows per head in the fused matrix.

nemo_automodel.components.models.ernie5_5.checkpoint_layout.DSAKVLayout.__post_init__() -> None

Reject a layout with a dimension that is not positive.

nemo_automodel.components.models.ernie5_5.checkpoint_layout._checked_ids(
expert_ids: collections.abc.Sequence[int]
) -> tuple[int, ...]

Return the expert indices as a tuple after validating them.

Parameters:

expert_ids
Sequence[int]

Indices of routed experts.

Returns: tuple[int, ...]

The indices in the given order.

Raises:

  • ValueError: If the sequence is empty, an index is not a non-negative integer or an index repeats.
nemo_automodel.components.models.ernie5_5.checkpoint_layout._layer_layout_and_experts(

Return what the decoder-layer conversions need besides the kinds of one planned layer.

Parameters:

config
Ernie5_5Config

Model configuration; supplies the DSA K/V dimensions and the routed expert count.

layer
Ernie5_5LayerPlan

Planned attention and feed-forward kinds of the layer.

Returns: DSAKVLayout | None

The K/V layout of a DSA layer (None for an HCA layer) and the indices of all routed experts in

nemo_automodel.components.models.ernie5_5.checkpoint_layout._pop_prefix(
remaining: dict[str, torch.Tensor],
prefix: str
) -> dict[str, torch.Tensor]

Remove every entry whose name starts with prefix and return them without the prefix.

Parameters:

remaining
dict[str, torch.Tensor]

Names mapped to tensors of any shape. Matching entries are removed in place.

prefix
str

Name prefix to select and strip.

Returns: dict[str, torch.Tensor]

The removed entries keyed by the rest of their names, in their original order. Tensors are the

nemo_automodel.components.models.ernie5_5.checkpoint_layout._put_prefix(
result: dict[str, torch.Tensor],
prefix: str,
state: collections.abc.Mapping[str, torch.Tensor]
) -> None

Add every entry of state to result under prefix + name.

Parameters:

result
dict[str, torch.Tensor]

Names mapped to tensors; updated in place.

prefix
str

Prefix prepended to every name of state.

state
Mapping[str, torch.Tensor]

Names mapped to tensors of any shape. Tensors are stored as given.

nemo_automodel.components.models.ernie5_5.checkpoint_layout._require_common_keys(
state: collections.abc.Mapping[str, torch.Tensor],
expected: set[str],
description: str
) -> None

Require exactly the names of the eight model-level tensors.

Parameters:

state
Mapping[str, torch.Tensor]

Names mapped to tensors of any shape.

expected
set[str]

The eight names that must be present, with no others.

description
str

What the names are, for the message.

Raises:

  • ValueError: If a name is missing or unexpected. The message lists the missing names and the first five unexpected ones; the keys of an MTP block, which a decoder-only conversion does not map, end here.
nemo_automodel.components.models.ernie5_5.checkpoint_layout._split_fused_kv_of(
combined: torch.Tensor,
) -> tuple[torch.Tensor, torch.Tensor]

Split a complete released fused K/V projection after checking its shape.

Parameters:

combined
torch.Tensor

Tensor of shape [heads * (key_channels + value_channels), latent].

layout
DSAKVLayout

Per-head dimensions of the fused K/V projection.

Returns: torch.Tensor

Keys of shape [heads * latent, key_channels] and values of shape [heads * value_channels, latent], as

Raises:

  • ValueError: If the fused matrix does not have the layout’s shape.
nemo_automodel.components.models.ernie5_5.checkpoint_layout._validate_direct(
state: collections.abc.Mapping[str, torch.Tensor],
vocab: int,
hidden: int
) -> None

Check the shapes of the five model-level tensors that keep their released layout.

Parameters:

state
Mapping[str, torch.Tensor]

Mapping that contains the Hugging Face names model.embed_tokens.weight [vocab, hidden], lm_head.weight [vocab, hidden], model.norm.weight [hidden], model.lm_head.multimax_ranges [4] and model.lm_head.multimax_ts [4]. Other entries are ignored.

vocab
int

Vocabulary size.

hidden
int

Hidden size.

Raises:

  • ValueError: If vocab or hidden is not positive or one of the tensors has another shape.
nemo_automodel.components.models.ernie5_5.checkpoint_layout._validate_keys(
state: collections.abc.Mapping[str, torch.Tensor],
expected: set[str]
) -> None

Require exactly the expected names of one mHC unit and FP32 tensors.

Parameters:

state
Mapping[str, torch.Tensor]

Relative mHC names mapped to tensors of any shape.

expected
set[str]

Names that must be present, with no others.

Raises:

  • ValueError: If a name is missing or unexpected, or a tensor is not FP32.
nemo_automodel.components.models.ernie5_5.checkpoint_layout._validate_kinds(
attention: str,
ffn: str,
expert_ids: collections.abc.Sequence[int] | None
) -> None

Check that the layer kinds are known and carry exactly the arguments they need.

Parameters:

attention
str

"hca" or "dsa".

ffn
str

"dense" or "moe".

layout
DSAKVLayout | None

K/V layout; required for DSA and not accepted for HCA.

expert_ids
Sequence[int] | None

Routed experts; required for MoE and not accepted for a dense feed-forward.

Raises:

  • ValueError: If a kind is unknown or an argument is missing or superfluous.
nemo_automodel.components.models.ernie5_5.checkpoint_layout.common_tensor_specs(
vocab: int,
hidden: int,
streams: int
) -> dict[str, tuple[str, tuple[int, ...]]]

Return the eight model-level tensors as native name -> (Hugging Face name, native shape).

The hyper-head matrix is the only one whose released layout differs from the native layout: the checkpoint stores hc_head_fn as [streams * hidden, streams], the transpose of hc_fn.

Parameters:

vocab
int

Vocabulary size.

hidden
int

Hidden size.

streams
int

Number of residual streams.

Returns: dict[str, tuple[str, tuple[int, ...]]]

Native names, relative to the inner model, mapped to the Hugging Face name and the native shape.

Return the per-head dimensions of the fused K/V projection of a configuration’s DSA layers.

Parameters:

config
Ernie5_5Config

Model configuration; supplies hybrid_mla_num_attention_heads, hybrid_mla_kv_lora_rank, hybrid_mla_qk_nope_head_dim and hybrid_mla_v_head_dim.

Returns: DSAKVLayout

The head count, the latent width and the key and value rows per head.

Raises:

  • ValueError: If a dimension is not positive.
nemo_automodel.components.models.ernie5_5.checkpoint_layout.install_fused_kv(
combined: torch.Tensor,
keys: torch.Tensor,
values: torch.Tensor,
) -> None

Split a released fused K/V projection and write the two parts into the native projections in place.

This completes a checkpoint read into an unsharded model: the fused projection is the one released tensor of a DSA layer that cannot be a view of model storage.

Parameters:

combined
torch.Tensor

Tensor of shape [heads * (key_channels + value_channels), latent] in the released layout.

keys
torch.Tensor

Native key projection of shape [heads * latent, key_channels]; overwritten in place, with a cast to its dtype.

values
torch.Tensor

Native value projection of shape [heads * value_channels, latent]; overwritten in place, with a cast to its dtype.

layout
DSAKVLayout

Per-head dimensions of the fused K/V projection.

Raises:

  • ValueError: If the fused matrix does not have the layout’s shape.
nemo_automodel.components.models.ernie5_5.checkpoint_layout.map_common_from_hf(
state: collections.abc.Mapping[str, torch.Tensor],
vocab: int,
hidden: int,
streams: int
) -> dict[str, torch.Tensor]

Map the eight model-level Hugging Face tensors to native names.

Parameters:

state
Mapping[str, torch.Tensor]

Exactly the eight released tensors model.embed_tokens.weight [vocab, hidden], lm_head.weight [vocab, hidden], model.norm.weight [hidden], model.lm_head.multimax_ranges [4], model.lm_head.multimax_ts [4], model.mhc_contract.hc_head_fn [streams * hidden, streams], model.mhc_contract.hc_head_base [streams] and model.mhc_contract.hc_head_scale [1]. The three hyper-head tensors must be FP32; the dtype of the others is not checked.

vocab
int

Vocabulary size.

hidden
int

Hidden size.

streams
int

Number of residual streams.

Returns: dict[str, torch.Tensor]

Native tensors named relative to the inner model: embed_tokens.weight [vocab, hidden],

Raises:

  • ValueError: If the key set differs from the eight names, a dimension is not positive, a shape does not match or a hyper-head tensor is not FP32.
nemo_automodel.components.models.ernie5_5.checkpoint_layout.map_common_to_hf(
state: collections.abc.Mapping[str, torch.Tensor],
vocab: int,
hidden: int,
streams: int
) -> dict[str, torch.Tensor]

Restore the eight model-level Hugging Face tensors from their native names.

Parameters:

state
Mapping[str, torch.Tensor]

Exactly the eight native tensors, named relative to the inner model: embed_tokens.weight [vocab, hidden], lm_head.weight [vocab, hidden], norm.weight [hidden], multimax.multimax_ranges [4], multimax.multimax_ts [4], hyper_head.hc_fn [streams, streams * hidden], hyper_head.hc_base [streams] and hyper_head.hc_scale [1]. The three hyper-head tensors must be FP32; the dtype of the others is not checked.

vocab
int

Vocabulary size.

hidden
int

Hidden size.

streams
int

Number of residual streams.

Returns: dict[str, torch.Tensor]

The released tensors in the layouts listed for map_common_from_hf.

Raises:

  • ValueError: If the key set differs from the eight names, a dimension is not positive, a shape does not match or a hyper-head tensor is not FP32.
nemo_automodel.components.models.ernie5_5.checkpoint_layout.map_decoder_from_hf(
state: collections.abc.Mapping[str, torch.Tensor],
attention: typing.Literal['hca', 'dsa'],
ffn: typing.Literal['dense', 'moe'],
expert_ids: collections.abc.Sequence[int] | None = None
) -> dict[str, torch.Tensor]

Map relative HF keys for one decoder layer to native block keys.

Selected experts are packed eagerly.

Parameters:

state
Mapping[str, torch.Tensor]

Complete tensors of one layer, named relative to model.layers.{index}: input_layernorm.weight and post_attention_layernorm.weight (any shape, passed through); self_attention_hyper_connection.* and mlp_hyper_connection.* in the layout of map_mhc_from_hf; self_attn.* in the layout of map_hca_from_hf or map_dsa_from_hf; and either mlp.gate_proj.weight, mlp.up_proj.weight and mlp.down_proj.weight (any shape, passed through) or block_sparse_moe.* in the layout of map_moe_from_hf.

attention
Literal['hca', 'dsa']

Attention kind of the layer.

ffn
Literal['dense', 'moe']

Feed-forward kind of the layer.

layout
DSAKVLayout | NoneDefaults to None

K/V layout; required for DSA and not accepted for HCA.

expert_ids
Sequence[int] | NoneDefaults to None

Routed experts to pack, in the order of the native expert axis; required for MoE and not accepted for a dense feed-forward.

Returns: dict[str, torch.Tensor]

Native tensors named relative to one decoder block: the two norm weights as given, attn_hc.*

Raises:

  • ValueError: If the kinds and their arguments are inconsistent, a key is missing or unexpected, or a component conversion rejects its tensors.
nemo_automodel.components.models.ernie5_5.checkpoint_layout.map_decoder_to_hf(
state: collections.abc.Mapping[str, torch.Tensor],
attention: typing.Literal['hca', 'dsa'],
ffn: typing.Literal['dense', 'moe'],
expert_ids: collections.abc.Sequence[int] | None = None
) -> dict[str, torch.Tensor]

Restore one native decoder block to relative HF decoder-layer keys.

Parameters:

state
Mapping[str, torch.Tensor]

Complete tensors of one decoder block, named relative to it: input_layernorm.weight and post_attention_layernorm.weight (any shape, passed through); attn_hc.* and ffn_hc.* in the layout of map_mhc_to_hf; attention.* in the layout of map_hca_to_hf or map_dsa_to_hf; and feed_forward.*, either the three dense projection weights (any shape, passed through) or the layout of map_moe_to_hf.

attention
Literal['hca', 'dsa']

Attention kind of the layer.

ffn
Literal['dense', 'moe']

Feed-forward kind of the layer.

layout
DSAKVLayout | NoneDefaults to None

K/V layout; required for DSA and not accepted for HCA.

expert_ids
Sequence[int] | NoneDefaults to None

Routed experts, one per entry of the native expert axis and in its order; required for MoE and not accepted for a dense feed-forward.

Returns: dict[str, torch.Tensor]

Released tensors named relative to model.layers.{index}, in the layouts accepted by

Raises:

  • ValueError: If the kinds and their arguments are inconsistent, a key is missing or unexpected, or a component conversion rejects its tensors.
nemo_automodel.components.models.ernie5_5.checkpoint_layout.map_dsa_from_hf(
state: collections.abc.Mapping[str, torch.Tensor],
) -> dict[str, torch.Tensor]

Convert all relative attention keys of one Hugging Face DSA layer to native keys.

Parameters:

state
Mapping[str, torch.Tensor]

Exactly the fifteen renamed suffixes relative to a single self_attn, mapped to tensors of any shape and dtype (not checked, not copied), plus kv_b_proj.weight of shape [heads * (key_channels + value_channels), latent].

layout
DSAKVLayout

Per-head dimensions of the fused K/V projection.

Returns: dict[str, torch.Tensor]

The renamed tensors as given, plus k_b_proj [heads * latent, key_channels] and v_b_proj

Raises:

  • ValueError: If a key is missing or unexpected, or the fused matrix does not have the layout’s shape.
nemo_automodel.components.models.ernie5_5.checkpoint_layout.map_dsa_to_hf(
state: collections.abc.Mapping[str, torch.Tensor],
) -> dict[str, torch.Tensor]

Restore relative HF attention keys, retaining input tensor dtypes.

Parameters:

state
Mapping[str, torch.Tensor]

Exactly the fifteen renamed native names of one DSA attention, mapped to tensors of any shape and dtype (not checked, not copied), plus k_b_proj [heads * latent, key_channels] and v_b_proj [heads * value_channels, latent] with one common dtype.

layout
DSAKVLayout

Per-head dimensions of the fused K/V projection.

Returns: dict[str, torch.Tensor]

The renamed tensors as given, plus a new kv_b_proj.weight of shape

Raises:

  • ValueError: If a key is missing or unexpected, the K/V projections do not have the layout’s shapes or their dtypes differ.
nemo_automodel.components.models.ernie5_5.checkpoint_layout.map_hca_from_hf(
state: collections.abc.Mapping[str, torch.Tensor]
) -> dict[str, torch.Tensor]

Rename a complete single HCA layer’s relative keys without copying tensors.

Parameters:

state
Mapping[str, torch.Tensor]

Exactly the fifteen released suffixes relative to a single self_attn, mapped to tensors of any shape and dtype. Shapes are not checked and no layout is changed.

Returns: dict[str, torch.Tensor]

The same tensor objects under the relative names expected by Ernie5_5HCAAttention.

Raises:

  • ValueError: If any expected key is missing or an unexpected key is present.
nemo_automodel.components.models.ernie5_5.checkpoint_layout.map_hca_to_hf(
state: collections.abc.Mapping[str, torch.Tensor]
) -> dict[str, torch.Tensor]

Restore a complete HCA component’s relative released key names.

Parameters:

state
Mapping[str, torch.Tensor]

Exactly the fifteen native names of one Ernie5_5HCAAttention, mapped to tensors of any shape and dtype. Shapes are not checked and no layout is changed.

Returns: dict[str, torch.Tensor]

The same tensor objects under the released suffixes relative to a single self_attn.

Raises:

  • ValueError: If any expected key is missing or an unexpected key is present.
nemo_automodel.components.models.ernie5_5.checkpoint_layout.map_hyper_head_from_hf(
state: collections.abc.Mapping[str, torch.Tensor],
streams: int,
hidden: int
) -> dict[str, torch.Tensor]

Map the released hyper-head tensors to the native stream-contraction module.

Parameters:

state
Mapping[str, torch.Tensor]

Exactly hc_head_fn [streams * hidden, streams], hc_head_base [streams] and hc_head_scale [1], all FP32, named relative to model.mhc_contract.

streams
int

Number of residual streams.

hidden
int

Hidden size.

Returns: dict[str, torch.Tensor]

hc_fn [streams, streams * hidden], the transpose of hc_head_fn made contiguous, and

Raises:

  • ValueError: If the key set differs, a dimension is not positive, a shape does not match or a tensor is not FP32.
nemo_automodel.components.models.ernie5_5.checkpoint_layout.map_hyper_head_to_hf(
state: collections.abc.Mapping[str, torch.Tensor],
streams: int,
hidden: int
) -> dict[str, torch.Tensor]

Restore the released hyper-head tensors from the native stream-contraction module.

Parameters:

state
Mapping[str, torch.Tensor]

Exactly hc_fn [streams, streams * hidden], hc_base [streams] and hc_scale [1], all FP32.

streams
int

Number of residual streams.

hidden
int

Hidden size.

Returns: dict[str, torch.Tensor]

hc_head_fn [streams * hidden, streams], a transposed view of hc_fn, and hc_head_base

Raises:

  • ValueError: If the key set differs, a dimension is not positive, a shape does not match or a tensor is not FP32.
nemo_automodel.components.models.ernie5_5.checkpoint_layout.map_mhc_from_hf(
state: collections.abc.Mapping[str, torch.Tensor]
) -> dict[str, torch.Tensor]

Convert the five released tensors of one mHC unit to the native layout.

The checkpoint stores the projection as [inputs, outputs] with three one-element alphas; the module stores it as [outputs, inputs] with one scale vector. In the model inputs is streams * hidden and outputs is (2 + streams) * streams; this function only checks that projection and bias agree.

Parameters:

state
Mapping[str, torch.Tensor]

Exactly mapping_proj.weight [inputs, outputs], bias [outputs], alpha_pre [1], alpha_post [1] and alpha_res [1], all FP32, named relative to one hyper-connection.

Returns: dict[str, torch.Tensor]

fn [outputs, inputs], a transposed view of the projection; base [outputs], the input bias;

Raises:

  • ValueError: If the key set differs, a tensor is not FP32, the projection is not a matrix, the bias does not match the projection or an alpha does not have shape [1].
nemo_automodel.components.models.ernie5_5.checkpoint_layout.map_mhc_to_hf(
state: collections.abc.Mapping[str, torch.Tensor]
) -> dict[str, torch.Tensor]

Convert one native mHC unit back to the five released tensors.

Parameters:

state
Mapping[str, torch.Tensor]

Exactly fn [outputs, inputs], base [outputs] and scale [3], all FP32.

Returns: dict[str, torch.Tensor]

mapping_proj.weight [inputs, outputs], a transposed view of fn; bias [outputs], the

Raises:

  • ValueError: If the key set differs, a tensor is not FP32, fn is not a matrix, base does not match fn or scale does not have shape [3].
nemo_automodel.components.models.ernie5_5.checkpoint_layout.map_model_from_hf(
state: collections.abc.Mapping[str, torch.Tensor],
) -> dict[str, torch.Tensor]

Map exactly a decoder-only HF state dict to the native names of the inner model.

All routed experts of every MoE layer are packed in memory, so this is meant for small checkpoints and for shape-only tensors on the meta device.

Parameters:

state
Mapping[str, torch.Tensor]

Complete, unsharded released tensors: model.layers.{index}.* for every decoder layer in the layouts of map_decoder_from_hf, and the eight model-level tensors in the layouts of map_common_from_hf.

config
Ernie5_5Config

Model configuration without MTP layers.

Returns: dict[str, torch.Tensor]

Native tensors named relative to the inner model: layers.{index}.* in the layouts returned by

Raises:

  • NotImplementedError: If the configuration has MTP layers.
  • ValueError: If a key is missing or unexpected (MTP and unknown keys are rejected, not dropped) or a component conversion rejects its tensors.
nemo_automodel.components.models.ernie5_5.checkpoint_layout.map_model_to_hf(
state: collections.abc.Mapping[str, torch.Tensor],
) -> dict[str, torch.Tensor]

Map exactly a decoder-only native state dict of the inner model back to HF names.

Parameters:

state
Mapping[str, torch.Tensor]

Complete, unsharded native tensors named relative to the inner model: layers.{index}.* for every decoder layer in the layouts of map_decoder_to_hf, and the eight model-level tensors in the layouts of map_common_to_hf.

config
Ernie5_5Config

Model configuration without MTP layers.

Returns: dict[str, torch.Tensor]

Released tensors: model.layers.{index}.* in the layouts returned by map_decoder_to_hf and

Raises:

  • NotImplementedError: If the configuration has MTP layers.
  • ValueError: If a key is missing or unexpected or a component conversion rejects its tensors.
nemo_automodel.components.models.ernie5_5.checkpoint_layout.map_moe_from_hf(
state: collections.abc.Mapping[str, torch.Tensor],
expert_ids: collections.abc.Sequence[int]
) -> dict[str, torch.Tensor]

Map non-expert keys and selected experts of one HF MoE layer.

Parameters:

state
Mapping[str, torch.Tensor]

Exactly the ten released non-expert tensors of one block_sparse_moe (any shape and dtype; not checked, not copied) and, for every selected expert, experts.{id}.w1.weight [intermediate, latent], experts.{id}.w2.weight [latent, intermediate] and experts.{id}.w3.weight [intermediate, latent].

expert_ids
Sequence[int]

Unique non-negative expert indices, in the order of the native expert axis.

Returns: dict[str, torch.Tensor]

The non-expert tensors as given under their native names, plus experts.gate_and_up_projs

Raises:

  • ValueError: If the indices are invalid, a key is missing or unexpected, or the experts cannot be packed.
nemo_automodel.components.models.ernie5_5.checkpoint_layout.map_moe_to_hf(
state: collections.abc.Mapping[str, torch.Tensor],
expert_ids: collections.abc.Sequence[int]
) -> dict[str, torch.Tensor]

Restore a local MoE’s grouped experts and non-expert keys to HF names.

Parameters:

state
Mapping[str, torch.Tensor]

Exactly the ten native non-expert tensors of one MoE (any shape and dtype; not checked, not copied), experts.gate_and_up_projs [experts, latent, 2 * intermediate] and experts.down_projs [experts, intermediate, latent].

expert_ids
Sequence[int]

Unique non-negative expert indices, one per entry of the expert axis and in its order.

Returns: dict[str, torch.Tensor]

The non-expert tensors as given under their released names, plus the three per-expert matrices

Raises:

  • ValueError: If the indices are invalid, a key is missing or unexpected, or the grouped tensors do not match the indices.
nemo_automodel.components.models.ernie5_5.checkpoint_layout.pack_experts(
state: collections.abc.Mapping[str, torch.Tensor],
expert_ids: collections.abc.Sequence[int]
) -> tuple[torch.Tensor, torch.Tensor]

Pack selected HF experts in the supplied order into grouped matrices.

Only the selected experts are allocated.

Parameters:

state
Mapping[str, torch.Tensor]

Mapping that contains, for every selected expert, experts.{id}.w1.weight (gate projection) [intermediate, latent], experts.{id}.w2.weight (down projection) [latent, intermediate] and experts.{id}.w3.weight (up projection) [intermediate, latent]. The selected tensors must share one dtype and one device. Other entries are ignored.

expert_ids
Sequence[int]

Unique non-negative expert indices. Their order is the order along the expert axis of the results.

Returns: torch.Tensor

Two new tensors with the dtype and device of the inputs: gate/up of shape

Raises:

  • ValueError: If the indices are invalid, a selected key is missing, the first gate projection is not a matrix, the shapes of an expert do not match, or dtypes or devices differ.
nemo_automodel.components.models.ernie5_5.checkpoint_layout.split_fused_kv(
combined: torch.Tensor,
heads: int,
latent: int,
key_channels: int,
value_channels: int
) -> tuple[torch.Tensor, torch.Tensor]

Split the released fused K/V projection into the separate native projections.

The fused matrix stores, for each head in turn, key_channels key rows followed by value_channels value rows, every row of length latent.

Parameters:

combined
torch.Tensor

Tensor of shape [heads * (key_channels + value_channels), latent]. It may be the global matrix or one rank’s shard of whole heads, with heads counting the heads it holds.

heads
int

Number of attention heads held by combined.

latent
int

Latent width of the K/V projection.

key_channels
int

Key rows per head.

value_channels
int

Value rows per head.

Returns: torch.Tensor

Keys of shape [heads * latent, key_channels], each head’s key block transposed, and values of

nemo_automodel.components.models.ernie5_5.checkpoint_layout.unpack_experts(
gate_up: torch.Tensor,
down: torch.Tensor,
expert_ids: collections.abc.Sequence[int]
) -> dict[str, torch.Tensor]

Restore selected grouped experts to their HF per-expert matrices.

Parameters:

gate_up
torch.Tensor

Tensor of shape [experts, latent, 2 * intermediate]. The last axis holds the transposed gate projection followed by the transposed up projection.

down
torch.Tensor

Tensor of shape [experts, intermediate, latent] with the dtype and device of gate_up.

expert_ids
Sequence[int]

Unique non-negative expert indices, one per entry of the expert axis and in its order.

Returns: dict[str, torch.Tensor]

For every index, experts.{id}.w1.weight [intermediate, latent], experts.{id}.w2.weight

Raises:

  • ValueError: If the indices are invalid, gate_up does not have three axes with an even last axis and one entry per index, or down does not match it in shape, dtype or device.
nemo_automodel.components.models.ernie5_5.checkpoint_layout.COMMON_KEYS = frozenset(_COMMON_NAMES)
nemo_automodel.components.models.ernie5_5.checkpoint_layout.DSA_HF_TO_NATIVE = {'q_a_proj.weight': 'q_a_proj.weight', 'q_a_layernorm.weight': 'q_a_layernorm.we...
nemo_automodel.components.models.ernie5_5.checkpoint_layout.HCA_HF_TO_NATIVE = {'linear_q_down_proj.weight': 'wq_a.weight', 'linear_q_up_proj.weight': 'wq_b.we...
nemo_automodel.components.models.ernie5_5.checkpoint_layout.MOE_HF_TO_NATIVE = {'gate.weight': 'gate.weight', 'gate.weight_1': 'gate.weight_1', 'gate.routed_sc...
nemo_automodel.components.models.ernie5_5.checkpoint_layout._COMMON_NAMES = {native: hf for native, (hf, _shape) in (common_tensor_specs(vocab=1, hidden=1, ...
nemo_automodel.components.models.ernie5_5.checkpoint_layout._DENSE = {'gate_proj.weight', 'up_proj.weight', 'down_proj.weight'}
nemo_automodel.components.models.ernie5_5.checkpoint_layout._DIRECT = {hf: native for native, hf in (_COMMON_NAMES.items()) if not native.startswith('...
nemo_automodel.components.models.ernie5_5.checkpoint_layout._HEAD_KEYS = {(hf.removeprefix(_HEAD_PREFIX)) for native, hf in (_COMMON_NAMES.items()) if na...
nemo_automodel.components.models.ernie5_5.checkpoint_layout._HEAD_PREFIX = 'model.mhc_contract.'
nemo_automodel.components.models.ernie5_5.checkpoint_layout._HF_KEYS = {'mapping_proj.weight', 'bias', 'alpha_pre', 'alpha_post', 'alpha_res'}
nemo_automodel.components.models.ernie5_5.checkpoint_layout._NATIVE_KEYS = {'fn', 'base', 'scale'}
nemo_automodel.components.models.ernie5_5.checkpoint_layout._NORMS = set(_NORM_KEYS)
nemo_automodel.components.models.ernie5_5.checkpoint_layout._NORM_KEYS = ('post_attention_layernorm.weight', 'input_layernorm.weight')