> This page is for version 0.4.0 · 26.04.
> For other versions, use one of these documentation indexes:
> - Nightly (default): https://docs.nvidia.com/nemo/automodel/nightly/llms.txt
> - Latest: https://docs.nvidia.com/nemo/automodel/latest/llms.txt
> - 0.5.0 · 26.06: https://docs.nvidia.com/nemo/automodel/v0.5/llms.txt
> - 0.4.0 · 26.04: https://docs.nvidia.com/nemo/automodel/v0.4/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.models.muse_spark.hf_state_dict_adapter

Checkpoint adapter for the nested Hugging Face Muse Spark Open release.

## Module Contents

### Classes

| Name                                                                                                                            | Description                                                               |
| ------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- |
| [`MuseSparkHFStateDictAdapter`](#nemo_automodel-components-models-muse_spark-hf_state_dict_adapter-MuseSparkHFStateDictAdapter) | Load the HF release through rank-local, predominantly direct DCP storage. |

### Functions

| Name                                                                                                          | Description                                                         |
| ------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
| [`_hf_key_to_native`](#nemo_automodel-components-models-muse_spark-hf_state_dict_adapter-_hf_key_to_native)   | Map one HF release name to the native model hierarchy.              |
| [`_native_key_to_hf`](#nemo_automodel-components-models-muse_spark-hf_state_dict_adapter-_native_key_to_hf)   | Map one native text, vision, or audio state name to the HF release. |
| [`_permute_rope_rows`](#nemo_automodel-components-models-muse_spark-hf_state_dict_adapter-_permute_rope_rows) | Convert adjacent-pair and split-half RoPE projection rows.          |

### Data

[`_AUDIO_BLOCK_SUFFIXES`](#nemo_automodel-components-models-muse_spark-hf_state_dict_adapter-_AUDIO_BLOCK_SUFFIXES)

[`_EXACT_KEYS`](#nemo_automodel-components-models-muse_spark-hf_state_dict_adapter-_EXACT_KEYS)

[`_HF_AUDIO_BLOCK_SUFFIXES`](#nemo_automodel-components-models-muse_spark-hf_state_dict_adapter-_HF_AUDIO_BLOCK_SUFFIXES)

[`_HF_EXACT_KEYS`](#nemo_automodel-components-models-muse_spark-hf_state_dict_adapter-_HF_EXACT_KEYS)

[`_HF_TEXT_SUFFIXES`](#nemo_automodel-components-models-muse_spark-hf_state_dict_adapter-_HF_TEXT_SUFFIXES)

[`_HF_VISION_SUFFIXES`](#nemo_automodel-components-models-muse_spark-hf_state_dict_adapter-_HF_VISION_SUFFIXES)

[`_TEXT_SUFFIXES`](#nemo_automodel-components-models-muse_spark-hf_state_dict_adapter-_TEXT_SUFFIXES)

[`_VISION_SUFFIXES`](#nemo_automodel-components-models-muse_spark-hf_state_dict_adapter-_VISION_SUFFIXES)

### API

```python
class nemo_automodel.components.models.muse_spark.hf_state_dict_adapter.MuseSparkHFStateDictAdapter()
```

**Bases:** [MuseSparkStateDictAdapter](/nemo-automodel/nemo_automodel/components/models/muse_spark/state_dict_adapter#nemo_automodel-components-models-muse_spark-state_dict_adapter-MuseSparkStateDictAdapter)

Load the HF release through rank-local, predominantly direct DCP storage.

Expert transposes are views into final native storage, including without EP.
Only Q/K row permutations allocate, bounded by the Q/K projections owned by
this stage and at most one gathered projection during conversion. No full
expert tensor is rebuilt after a direct load.

**`supports_low_memory_dcp_load`** `bool`

Enable direct destinations only for the supported dense expert layouts.

---

```python
nemo_automodel.components.models.muse_spark.hf_state_dict_adapter.MuseSparkHFStateDictAdapter._rope_head_dim(
    native_key: str
) -> int | None
```

Return the head width for text/vision Q/K weights and vision Q/K biases.

```python
nemo_automodel.components.models.muse_spark.hf_state_dict_adapter.MuseSparkHFStateDictAdapter.convert_single_tensor_to_hf(
    fqn: str,
    tensor: torch.Tensor,
    kwargs: str | bool | None = {}
) -> list[tuple[str, torch.Tensor]]
```

Convert one native tensor into an HF checkpoint tensor.

**Parameters:**

**`fqn`** `str`

Native state name.

---

**`tensor`** `torch.Tensor`

Expert \[experts, in, out], router \[experts, hidden],
projection \[out, in], bias/norm \[out], or embedding \[vocab, hidden].
DTensors retain their global shapes and parameter placements.

---

**`**kwargs`** `str | bool | None` — default: \{}

Checkpointer's exclude_key_regex and for_checkpoint_load.

---

**Returns:** `list[tuple[str, torch.Tensor]]`

Zero or one (HF name, tensor) pairs. Expert \[experts, out, in]

```python
nemo_automodel.components.models.muse_spark.hf_state_dict_adapter.MuseSparkHFStateDictAdapter.from_hf(
    hf_state_dict: dict[str, torch.Tensor],
    device_mesh: torch.distributed.device_mesh.DeviceMesh | None = None,
    kwargs: str | bool | None = {}
) -> dict[str, torch.Tensor]
```

Convert HF text, vision and audio tensors to the native checkpoint.

**Parameters:**

**`hf_state_dict`** `dict[str, torch.Tensor]`

HF names mapped to experts \[experts, out, in],
routers \[experts, hidden], projections \[out, in], vectors \[out],
and embeddings \[vocab, hidden]. DCP destinations may be strided
model-storage views or DTensors with global shapes.

---

**`device_mesh`** `DeviceMesh | None` — default: None

Optional EP mesh used to slice unsharded expert input.
Already-sharded DTensor experts preserve their existing mesh.

---

**`**kwargs`** `str | bool | None` — default: \{}

Unused shared checkpoint adapter arguments.

---

**Returns:** `dict[str, torch.Tensor]`

Native names and tensor layouts. Expert transposes and unchanged

```python
nemo_automodel.components.models.muse_spark.hf_state_dict_adapter._hf_key_to_native(
    key: str
) -> str
```

Map one HF release name to the native model hierarchy.

```python
nemo_automodel.components.models.muse_spark.hf_state_dict_adapter._native_key_to_hf(
    key: str
) -> str | None
```

Map one native text, vision, or audio state name to the HF release.

```python
nemo_automodel.components.models.muse_spark.hf_state_dict_adapter._permute_rope_rows(
    tensor: torch.Tensor,
    head_dim: int,
    to_hf: bool
) -> torch.Tensor
```

Convert adjacent-pair and split-half RoPE projection rows.

**Parameters:**

**`tensor`** `torch.Tensor`

Projection \[heads \* head_dim, hidden] or bias \[heads \* head_dim].
A DTensor retains its global shape and arbitrary Shard/Replicate
placements. Rank-local row fragments need not span complete heads.

---

**`head_dim`** `int`

Width of a complete attention head.

---

**`to_hf`** `bool`

Convert adjacent pairs to split halves when true; invert otherwise.

---

**Returns:** `torch.Tensor`

Independent contiguous tensor with the input shape, dtype and device.

```python
nemo_automodel.components.models.muse_spark.hf_state_dict_adapter._AUDIO_BLOCK_SUFFIXES = {None: {f'{native}.{part}': f'{hf}.{part}' for native, hf in (('conv.conv', 'con...
```

```python
nemo_automodel.components.models.muse_spark.hf_state_dict_adapter._EXACT_KEYS = {'lm_head.weight': 'lm_head.weight', 'model.embed_tokens.weight': 'model.languag...
```

```python
nemo_automodel.components.models.muse_spark.hf_state_dict_adapter._HF_AUDIO_BLOCK_SUFFIXES = {value: key for key, value in (_AUDIO_BLOCK_SUFFIXES.items())}
```

```python
nemo_automodel.components.models.muse_spark.hf_state_dict_adapter._HF_EXACT_KEYS = {value: key for key, value in (_EXACT_KEYS.items())}
```

```python
nemo_automodel.components.models.muse_spark.hf_state_dict_adapter._HF_TEXT_SUFFIXES = {value: key for key, value in (_TEXT_SUFFIXES.items())}
```

```python
nemo_automodel.components.models.muse_spark.hf_state_dict_adapter._HF_VISION_SUFFIXES = {value: key for key, value in (_VISION_SUFFIXES.items())}
```

```python
nemo_automodel.components.models.muse_spark.hf_state_dict_adapter._TEXT_SUFFIXES = {'self_attn.pre_qkv_proj_norm.weight': 'input_layernorm.weight', 'self_attn.o_ga...
```

```python
nemo_automodel.components.models.muse_spark.hf_state_dict_adapter._VISION_SUFFIXES = {None: {f'attn.{proj}_proj.{part}': f'attn.{proj}_proj.{part}' for proj in ('q',...
```