core.dist_checkpointing.layer_key_aliases#
Load distributed checkpoints saved before a layer spec’s sharded_state_dict_keys_map.
A layer spec’s sharded_state_dict_keys_map renames a layer’s runtime module names to
canonical checkpoint names (e.g. input_layernorm. ->
self_attention.linear_qkv.layer_norm_). Checkpoints saved without that renaming store the
same tensors under the runtime names. Rather than changing the model, the load request is
retargeted at those names, so later saves still use the canonical ones.
Module Contents#
Functions#
Resolve layer-spec key aliases in a distributed checkpoint’s load request. |
Data#
API#
- core.dist_checkpointing.layer_key_aliases.logger#
‘getLogger(…)’
- core.dist_checkpointing.layer_key_aliases.remap_sharded_state_dict_for_load(
- models,
- state_dict,
- checkpoint_dir,
- sharded_strategy=None,
Resolve layer-spec key aliases in a distributed checkpoint’s load request.
Prefer the requested checkpoint key. If it is absent, try the corresponding runtime module name from the layer specs. Only the load template is changed; the models and their canonical save mappings stay untouched. Apply this after other load modifiers to the complete template, including model-keyed optimizer state.
sharded_strategyreads the checkpoint metadata; pass the strategy of the load that follows so that both share its metadata cache. Defaults to the default load strategy.