nemo_automodel.components.models.llava_onevision.state_dict_adapter

State dict adapter for LLaVA-OneVision-1.5.

HF on-disk safetensors layout (from lmms-lab/LLaVA-OneVision-1.5-): visual.{patch_embed,class_embedding,class_pos_emb,pre_layernorm,blocks.,merger.} model.{embed_tokens,layers.,norm} lm_head.weight

Applies the same regex rename HF does via _checkpoint_conversion_mapping: ^visual -> model.visual ^model(?!.(language_model|visual)) -> model.language_model

Module Contents

Classes

Name	Description
`LlavaOneVisionStateDictAdapter`	-

Functions

Name	Description
`_rename`	-

Data

_HF_TO_NEMO_RULES

_NEMO_TO_HF_RULES

API

class nemo_automodel.components.models.llava_onevision.state_dict_adapter.LlavaOneVisionStateDictAdapter(
    config: typing.Any = None,
    kwargs = {}
)

Bases: StateDictAdapter

nemo_automodel.components.models.llava_onevision.state_dict_adapter.LlavaOneVisionStateDictAdapter.convert_single_tensor_to_hf(
    fqn: str,
    tensor: typing.Any,
    kwargs = {}
) -> list[tuple[str, typing.Any]]

nemo_automodel.components.models.llava_onevision.state_dict_adapter.LlavaOneVisionStateDictAdapter.from_hf(
    hf_state_dict: dict[str, typing.Any],
    kwargs = {}
) -> dict[str, typing.Any]

nemo_automodel.components.models.llava_onevision.state_dict_adapter.LlavaOneVisionStateDictAdapter.to_hf(
    state_dict: dict[str, typing.Any],
    exclude_key_regex: typing.Optional[str] = None,
    kwargs = {}
) -> dict[str, typing.Any]

nemo_automodel.components.models.llava_onevision.state_dict_adapter._rename(
    key: str,
    rules
) -> str

nemo_automodel.components.models.llava_onevision.state_dict_adapter._HF_TO_NEMO_RULES = [(re.compile('^visual\\.'), 'model.visual.'), (re.compile('^model\\.(?!language_...

nemo_automodel.components.models.llava_onevision.state_dict_adapter._NEMO_TO_HF_RULES = [(re.compile('^model\\.visual\\.'), 'visual.'), (re.compile('^model\\.language_m...