Skip to navigation
ReferenceFull Library ReferenceNemo AutomodelNemo AutomodelComponentsModelsErnie5 5nemo_automodel.components.models.ernie5_5.state_dict_adapter

nemo_automodel.components.models.ernie5_5.state_dict_adapter

View as MarkdownOpen in Claude

State-dict adapter between the native ERNIE 5.5 model and the released Hugging Face checkpoint layout.

Ernie5_5StateDictAdapter is the object the checkpointer talks to. It serves an unsharded model with the conversions of checkpoint_layout, whose released tensors are views of the native ones, and a sharded model with the destinations and export views of checkpoint_io.

A checkpoint load is a session of two calls: to_hf(..., for_checkpoint_load=True) returns the destinations that the reader fills, and from_hf receives the same mapping back, installs what could not be read in place and returns the native state. The adapter keeps one session at a time and fails closed: after a session that was abandoned, replaced or failed, every later load or save raises, because the model may be partially overwritten.

The tensors of a PEFT export keep their names: the LoRA adapter tensors, and the modules that the export stores whole under the base_model.model. prefix (modules_to_save, such as an unfrozen output head). to_hf and from_hf return a state of such tensors alone as it is, without a session, and to_hf appends them to the converted base tensors of a state that also holds base tensors. In a load, from_hf raises ValueError unless the tensors it receives have the keys, shapes and dtypes of the PEFT tensors that to_hf(..., for_checkpoint_load=True) was given.

Limitations:

  • Decoder-only: the configuration must have num_nextn_predict_layers == 0 and untied embedding and output-head weights. MTP weights are neither loaded nor exported.
  • No quantized checkpoints, no v4_compatible metadata export and no streaming export of single tensors.
  • A sharded load or save needs the layouts listed in checkpoint_io. All ranks must make the same calls in the same order.
  • A load is not transactional: after a failure the model and the adapter must be discarded. Concurrent sessions and use from several threads are not supported.
  • An unsharded load through the checkpointer is supported only without a device mesh. Like a sharded load it reads into model storage; the fused K/V projection of each DSA layer is its only temporary.
  • A conversion from the released layout without a load session (from_hf on plain tensors) packs every routed expert into new tensors and so holds a second copy of the experts; it suits small checkpoints.
  • Supported stored dtypes of a checkpoint are BF16, FP16 and FP32. The adapter never sees the stored dtype, so it cannot reject another one: the reader would convert it silently (FP64 would be rounded).

Module Contents

Classes

NameDescription
Ernie5_5StateDictAdapterConvert between the native state of Ernie5_5_MoeForCausalLM and the released checkpoint layout.
_LoadDestinationsDestinations of one checkpoint load, keyed by Hugging Face name.
_SessionTokenIdentity of one load session; two tokens are never the same object.

Functions

NameDescription
_is_peft_keyReturn whether a state key names a tensor of a PEFT export: a LoRA adapter tensor, or a tensor of a module

Data

_MODEL_PREFIX

API

class nemo_automodel.components.models.ernie5_5.state_dict_adapter.Ernie5_5StateDictAdapter(
)

Bases: StateDictAdapter

Convert between the native state of Ernie5_5_MoeForCausalLM and the released checkpoint layout.

Native names carry the model. prefix of the registered class. For a sharded and for an unsharded state, to_hf returns destinations or export views that alias model storage, so that a checkpoint read writes the model in place and a distributed read or save moves each rank’s shard only.

Key planning (get_hf_state_dict_keys) is supported for the unsharded state only, which is how the framework uses it: once, before the model is sharded. It converts a shape-only copy of the state with to_hf. A sharded state is rejected, and like a save it raises while a load session is open and after a session was abandoned, replaced or failed.

Loading into an unsharded model is supported without a device mesh, which is what a run in one process passes. An unsharded state together with a device mesh is not supported: from_hf raises, and inside a load session the adapter then refuses all further work.

_supports_low_memory_dcp_load is set because a load allocates only small temporaries, also in one process: the fused K/V projection of each DSA layer and, for a sharded model, the three alphas of each mHC unit. Every other destination, the routed experts included, is model storage.

_active
ReferenceType[_LoadDestinations] | None = None
_ended
list[ReferenceType[Tensor]] = []
_peft_expected
dict[str, tuple[Size, dtype]] | None = None
_token
_SessionToken | None = None
nemo_automodel.components.models.ernie5_5.state_dict_adapter.Ernie5_5StateDictAdapter.__deepcopy__(
memo: dict[int, typing.Any]
) -> typing.Self

Return an adapter for a deep copy of the configuration that shares no load session with this one.

A model is copied together with its adapter. A copy made while a load session is open, or after one was abandoned, replaced or failed, refuses all further work: the copied model may be partially overwritten, and the open session belongs to the original.

Parameters:

memo
dict[int, Any]

The memo dictionary of copy.deepcopy.

Returns: Self

A new adapter, usable when this adapter is idle. It refuses the destinations of this adapter’s

nemo_automodel.components.models.ernie5_5.state_dict_adapter.Ernie5_5StateDictAdapter._add_prefix(
state: collections.abc.Mapping[str, torch.Tensor]
) -> dict[str, torch.Tensor]
staticmethod

Return the state under the names of the registered class.

Parameters:

state
Mapping[str, torch.Tensor]

Names relative to the inner model mapped to tensors of any shape.

Returns: dict[str, torch.Tensor]

The same tensor objects, in the same order, with the model. prefix.

nemo_automodel.components.models.ernie5_5.state_dict_adapter.Ernie5_5StateDictAdapter._holds_ended_destinations(
mapping: collections.abc.Mapping[str, torch.Tensor]
) -> bool

Return whether a mapping holds a tensor that was created as a destination of an ended session.

Such a mapping is a copy of finished destinations. Its tensors hold what an earlier load left in them, and the entries that alias model storage hold the current model values.

Parameters:

mapping
Mapping[str, torch.Tensor]

Hugging Face names mapped to plain tensors of any shape.

Returns: bool

True when at least one tensor object of the mapping is a view or a temporary of an ended session.

nemo_automodel.components.models.ernie5_5.state_dict_adapter.Ernie5_5StateDictAdapter._open_session(
) -> None

Make destinations the one active load session.

The adapter keeps only a weak link to the mapping. If the mapping is dropped before from_hf receives it, the session counts as abandoned and the adapter refuses all further work.

Parameters:

destinations
_LoadDestinations

Hugging Face names mapped to the destination tensors of the load, of any shape.

nemo_automodel.components.models.ernie5_5.state_dict_adapter.Ernie5_5StateDictAdapter._rename_feed_forward(
state: collections.abc.Mapping[str, torch.Tensor],
to_module_names: bool
) -> dict[str, torch.Tensor]

Rename the feed-forward tensors of every decoder layer between the two native spellings.

The eager conversions name the feed-forward feed_forward; the model registers it as mlp (dense) or moe (MoE).

Parameters:

state
Mapping[str, torch.Tensor]

Names relative to the inner model mapped to tensors of any shape.

to_module_names
bool

Rename layers.{index}.feed_forward.* to the module names when true, the module names to feed_forward when false.

Returns: dict[str, torch.Tensor]

The same tensor objects. The renamed tensors of each layer follow all other entries, layer by

Raises:

  • ValueError: If a layer has no feed-forward tensor under the source name or already has one under the target name.
nemo_automodel.components.models.ernie5_5.state_dict_adapter.Ernie5_5StateDictAdapter._strip_prefix(
state: collections.abc.Mapping[str, torch.Tensor]
) -> dict[str, torch.Tensor]
staticmethod

Return the state named relative to the inner model.

Parameters:

state
Mapping[str, torch.Tensor]

Native names with the model. prefix mapped to tensors of any shape.

Returns: dict[str, torch.Tensor]

The same tensor objects, in the same order, without the prefix.

Raises:

  • ValueError: If a name does not start with model..
nemo_automodel.components.models.ernie5_5.state_dict_adapter.Ernie5_5StateDictAdapter._unsharded_finishers(
native: collections.abc.Mapping[str, torch.Tensor],
targets: collections.abc.Mapping[str, torch.Tensor]
) -> list[collections.abc.Callable[[], None]]

Return the callbacks that complete a checkpoint read into an unsharded model.

A read fills the destinations in place, and all of them alias model storage except the fused K/V projection of each DSA layer. Each callback splits one filled fused projection into the native key and value projections.

Parameters:

native
Mapping[str, torch.Tensor]

Native tensors named relative to the inner model, in the shapes of the model’s parameters and buffers. The callbacks overwrite layers.{index}.attention.k_b_proj [heads * latent, key_channels] and layers.{index}.attention.v_b_proj [heads * value_channels, latent] of every DSA layer in place.

targets
Mapping[str, torch.Tensor]

Released Hugging Face names mapped to the destinations of the load. The callbacks read model.layers.{index}.self_attn.kv_b_proj.weight of shape [heads * (key_channels + value_channels), latent] of every DSA layer.

Returns: list[Callable[[], None]]

One callback per DSA layer, in layer order.

nemo_automodel.components.models.ernie5_5.state_dict_adapter.Ernie5_5StateDictAdapter._unsharded_from_hf(
hf_state_dict: collections.abc.Mapping[str, torch.Tensor],
device_mesh: torch.distributed.device_mesh.DeviceMesh | None
) -> dict[str, torch.Tensor]

Convert a complete unsharded state in the released layout to the native state.

Parameters:

hf_state_dict
Mapping[str, torch.Tensor]

Exactly the released tensors of the decoder-only model, in the released layouts; plain tensors.

device_mesh
DeviceMesh | None

Mesh of a distributed load; must be None. Loading into an unsharded model with a device mesh is not supported, because this conversion does not shard.

Returns: dict[str, torch.Tensor]

Native model. names mapped to tensors in the shapes of the model’s parameters and buffers.

Raises:

  • ValueError: If a device mesh is given, a tensor is missing or unexpected, or a component conversion rejects its tensors.
nemo_automodel.components.models.ernie5_5.state_dict_adapter.Ernie5_5StateDictAdapter._unsharded_to_hf(
state_dict: collections.abc.Mapping[str, torch.Tensor]
) -> dict[str, torch.Tensor]

Convert a complete unsharded native state to the released layout.

Parameters:

state_dict
Mapping[str, torch.Tensor]

Exactly the native tensors of the model under their model. names, in the shapes of the model’s parameters and buffers; plain tensors on any device, the meta device included.

Returns: dict[str, torch.Tensor]

Released Hugging Face names mapped to tensors in the released layouts, in the order of

Raises:

  • ValueError: If a name lacks the model. prefix, a tensor is missing or unexpected, or a component conversion rejects its tensors.
nemo_automodel.components.models.ernie5_5.state_dict_adapter.Ernie5_5StateDictAdapter.convert_single_tensor_to_hf(
fqn: str,
tensor: torch.Tensor,
kwargs: str | bool | torch.distributed.device_mesh.DeviceMesh | None = {}
) -> list[tuple[str, torch.Tensor]]

Reject the conversion of a single tensor.

The released layout fuses the key and value projections of a DSA layer and splits the grouped expert tensors, so one native tensor cannot be converted without the others.

Parameters:

fqn
str

Native name of the tensor.

tensor
torch.Tensor

Tensor of any shape.

**kwargs
str | bool | DeviceMesh | NoneDefaults to {}

Options of the caller; not read.

Raises:

  • NotImplementedError: Always.
nemo_automodel.components.models.ernie5_5.state_dict_adapter.Ernie5_5StateDictAdapter.from_hf(
hf_state_dict: dict[str, torch.Tensor],
device_mesh: torch.distributed.device_mesh.DeviceMesh | None = None,
kwargs: str | bool | None = {}
) -> dict[str, torch.Tensor]

Return the native state for tensors in the released layout, closing a load session if one is open.

Without a session, a complete mapping of plain tensors is converted on its own. Otherwise the mapping must be the object that to_hf(..., for_checkpoint_load=True) returned for the open session, with the same keys and the same tensor objects. Anything else makes the adapter refuse all further work: a copy of the destinations, made during the session or after it, the destinations of another adapter or of a session that has already ended, a mapping of DTensors or an empty mapping without a session.

Parameters:

hf_state_dict
dict[str, torch.Tensor]

Released Hugging Face names mapped to tensors in the released layouts: either the destinations of the active load session after the read, or, when no session is open, exactly the released tensors of the decoder-only model as plain tensors, none of which was a destination of an earlier session.

device_mesh
DeviceMesh | NoneDefaults to None

Mesh of a distributed load. A sharded session does not need it. An unsharded session and a conversion without a session require None: loading into an unsharded model with a device mesh is not supported.

**kwargs
str | bool | NoneDefaults to {}

Other options of the caller. None applies to this model; they are not read.

Returns: dict[str, torch.Tensor]

Native model. names mapped to tensors in the shapes of the model’s parameters and buffers. For

Raises:

  • RuntimeError: If the mapping is not the destinations of the active session, or an earlier session was abandoned, replaced or failed.
  • ValueError: If the keys or tensors of the destinations were exchanged, a device mesh is given for an unsharded conversion, or a conversion rejects its tensors.
nemo_automodel.components.models.ernie5_5.state_dict_adapter.Ernie5_5StateDictAdapter.get_hf_state_dict_keys(
state_dict: dict[str, torch.Tensor]
) -> list[str]

Return the Hugging Face keys of the unsharded model, in the order of an unsharded save.

The framework plans the keys once, before the model is sharded. A sharded state is rejected instead of planned: its export names only the experts that one rank holds and orders the keys differently, so it cannot stand in for the plan of the whole model.

Parameters:

state_dict
dict[str, torch.Tensor]

Exactly the native tensors of the unsharded model under their model. names, in the shapes of the model’s parameters and buffers; plain tensors on any device. Only shapes and dtypes are read.

Returns: list[str]

The released Hugging Face names, in the order in which to_hf returns them for an unsharded

Raises:

  • ValueError: If a tensor of the state is a DTensor, or the unsharded conversion rejects the state.
  • RuntimeError: If a load session is open, or an earlier session was abandoned, replaced or failed.
  • NotImplementedError: If the configuration has been changed to MTP layers or tied weights.
nemo_automodel.components.models.ernie5_5.state_dict_adapter.Ernie5_5StateDictAdapter.to_hf(
state_dict: dict[str, torch.Tensor],
quantization: bool = False,
for_checkpoint_load: bool = False,
v4_compatible: bool = False,
kwargs: str | bool | torch.distributed.device_mesh.DeviceMesh | None = {}
) -> dict[str, torch.Tensor]

Return the released layout of the native state, as load destinations or as tensors to save.

The state is treated as sharded when its tensors are DTensors. With for_checkpoint_load the result opens a load session that from_hf must close; without it the result is what a save writes.

Parameters:

state_dict
dict[str, torch.Tensor]

Exactly the native tensors of the model under their model. names, in the shapes of the model’s parameters and buffers. Either all plain tensors, or all DTensors in the layouts required by model_hf_dcp_load_destinations (load) and model_hf_distributed_export (save): for a DTensor the shapes are global and each rank holds its shard.

quantization
boolDefaults to False

Must be false; quantized checkpoints are not supported.

for_checkpoint_load
boolDefaults to False

Return destinations for a checkpoint read instead of tensors to save.

v4_compatible
boolDefaults to False

Must be false; the legacy metadata export is not supported.

**kwargs
str | bool | DeviceMesh | NoneDefaults to {}

Other options of the checkpointer (exclude_key_regex, device_mesh, legacy_paramwrapper_layout). They do not apply to this model and are not read.

Returns: dict[str, torch.Tensor]

Released Hugging Face names mapped to tensors in the released layouts.

Raises:

  • NotImplementedError: If quantization or v4_compatible is set, or the configuration has been changed to MTP layers or tied weights.
  • RuntimeError: If a load session is active, or an earlier session was abandoned, replaced or failed.
  • ValueError: If the state is empty (save), mixes plain tensors and DTensors (save), has a name without the model. prefix, or a conversion rejects its tensors.
  • TypeError: If a sharded state holds a plain tensor where a DTensor is required.
class nemo_automodel.components.models.ernie5_5.state_dict_adapter._LoadDestinations()

Bases: dict[str, Tensor]

Destinations of one checkpoint load, keyed by Hugging Face name.

The mapping owns the temporaries of the load and the callbacks that install them. The adapter holds it weakly, so dropping the mapping releases the temporaries.

created
tuple[ReferenceType[Tensor], ...]

Weak references to the destination tensors that are not tensors of the given state: the views and temporaries that the adapter created for this load.

finishers
tuple[Callable[[], None], ...]

Callbacks that install the temporaries into the native tensors after the read, in order.

native
dict[str, Tensor]

The native tensors that the load fills, named relative to the inner model: the tensors of the state that to_hf was given.

original
dict[str, Tensor]

The destinations as created, to detect a mapping whose keys or tensors were exchanged.

unsharded
bool

Whether the state is unsharded (plain tensors) rather than sharded (DTensors).

class nemo_automodel.components.models.ernie5_5.state_dict_adapter._SessionToken()

Identity of one load session; two tokens are never the same object.

nemo_automodel.components.models.ernie5_5.state_dict_adapter._is_peft_key(
name: str
) -> bool

Return whether a state key names a tensor of a PEFT export: a LoRA adapter tensor, or a tensor of a module that the export stores whole under the base_model.model. prefix.

nemo_automodel.components.models.ernie5_5.state_dict_adapter._MODEL_PREFIX = 'model.'