> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.models.muse_glimmer.vision

Native MuseGlimmer vision path copied from the canonical Transformers implementation.

## Module Contents

### Classes

| Name                                                                                                                         | Description                                           |
| ---------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------- |
| [`MuseGlimmerVisionAdapter`](#nemo_automodel-components-models-muse_glimmer-vision-MuseGlimmerVisionAdapter)                 | Canonical two-layer visual adapter.                   |
| [`MuseGlimmerVisionAttention`](#nemo_automodel-components-models-muse_glimmer-vision-MuseGlimmerVisionAttention)             | Canonical packed bidirectional vision attention.      |
| [`MuseGlimmerVisionBlock`](#nemo_automodel-components-models-muse_glimmer-vision-MuseGlimmerVisionBlock)                     | Canonical vision encoder layer.                       |
| [`MuseGlimmerVisionEncoder`](#nemo_automodel-components-models-muse_glimmer-vision-MuseGlimmerVisionEncoder)                 | Canonical processor-patch vision encoder.             |
| [`MuseGlimmerVisionMLP`](#nemo_automodel-components-models-muse_glimmer-vision-MuseGlimmerVisionMLP)                         | Canonical vision MLP.                                 |
| [`MuseGlimmerVisionRotaryEmbedding`](#nemo_automodel-components-models-muse_glimmer-vision-MuseGlimmerVisionRotaryEmbedding) | Canonical independent-frequency two-axis vision RoPE. |

### Functions

| Name                                                                                                                                       | Description                                                          |
| ------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------- |
| [`apply_rotary_pos_emb_vision`](#nemo_automodel-components-models-muse_glimmer-vision-apply_rotary_pos_emb_vision)                         | Apply the canonical split-half vision RoPE in float32.               |
| [`get_vision_bilinear_indices_and_weights`](#nemo_automodel-components-models-muse_glimmer-vision-get_vision_bilinear_indices_and_weights) | Copy the checkpoint's grid-sample-equivalent position interpolation. |
| [`get_vision_cu_seqlens`](#nemo_automodel-components-models-muse_glimmer-vision-get_vision_cu_seqlens)                                     | Return one packed-attention segment per frame.                       |
| [`get_vision_position_ids`](#nemo_automodel-components-models-muse_glimmer-vision-get_vision_position_ids)                                 | Build canonical block-major two-dimensional vision positions.        |
| [`get_vision_window_index`](#nemo_automodel-components-models-muse_glimmer-vision-get_vision_window_index)                                 | Copy the canonical window-attention permutation and segment lengths. |
| [`rotate_half`](#nemo_automodel-components-models-muse_glimmer-vision-rotate_half)                                                         | Rotate the two halves of the hidden dimension.                       |

### API

```python
class nemo_automodel.components.models.muse_glimmer.vision.MuseGlimmerVisionAdapter(
    config: nemo_automodel.components.models.muse_glimmer.config.MuseGlimmerConfig
)
```

**Bases:** `Module`

Canonical two-layer visual adapter.

**`act`** `= ACT2FN[config.projector_hidden_act]`

---

**`c_fc`**

---

**`c_proj`**

---

```python
nemo_automodel.components.models.muse_glimmer.vision.MuseGlimmerVisionAdapter.forward(
    x: torch.Tensor
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.muse_glimmer.vision.MuseGlimmerVisionAttention(
    config
)
```

**Bases:** `Module`

Canonical packed bidirectional vision attention.

**`dim`** `= config.hidden_size`

---

**`head_dim`** `= self.dim // self.num_heads`

---

**`k_proj`** `= nn.Linear(self.dim, self.dim, bias=True)`

---

**`num_heads`** `= config.num_attention_heads`

---

**`o_proj`** `= nn.Linear(self.dim, self.dim, bias=True)`

---

**`q_proj`** `= nn.Linear(self.dim, self.dim, bias=True)`

---

**`scaling`** `= self.head_dim ** -0.5`

---

**`v_proj`** `= nn.Linear(self.dim, self.dim, bias=True)`

---

```python
nemo_automodel.components.models.muse_glimmer.vision.MuseGlimmerVisionAttention.forward(
    hidden_states: torch.Tensor,
    cu_seqlens: torch.Tensor,
    position_embeddings: tuple[torch.Tensor, torch.Tensor]
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.muse_glimmer.vision.MuseGlimmerVisionBlock(
    config
)
```

**Bases:** `GradientCheckpointingLayer`

Canonical vision encoder layer.

**`attn`** `= MuseGlimmerVisionAttention(config)`

---

**`ln_1`** `= nn.LayerNorm(config.hidden_size, eps=1e-05)`

---

**`ln_2`** `= nn.LayerNorm(config.hidden_size, eps=1e-05)`

---

**`mlp`** `= MuseGlimmerVisionMLP(config)`

---

```python
nemo_automodel.components.models.muse_glimmer.vision.MuseGlimmerVisionBlock.forward(
    hidden_states: torch.Tensor,
    cu_seqlens: torch.Tensor,
    position_embeddings: tuple[torch.Tensor, torch.Tensor]
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.muse_glimmer.vision.MuseGlimmerVisionEncoder(
    config: nemo_automodel.components.models.muse_glimmer.config.MuseGlimmerConfig
)
```

**Bases:** `Module`

Canonical processor-patch vision encoder.

**`conv1_linear`**

---

**`ln_post`**

---

**`ln_pre`**

---

**`positional_embedding_vlm`**

---

**`rotary_emb`** `= MuseGlimmerVisionRotaryEmbedding(vision_config)`

---

**`transformer`**

---

```python
nemo_automodel.components.models.muse_glimmer.vision.MuseGlimmerVisionEncoder._pixel_shuffle(
    hidden_states: torch.Tensor,
    grid_thw: torch.Tensor
) -> torch.Tensor
```

```python
nemo_automodel.components.models.muse_glimmer.vision.MuseGlimmerVisionEncoder.forward(
    pixel_values: torch.Tensor,
    grid_thw: torch.Tensor
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.muse_glimmer.vision.MuseGlimmerVisionMLP(
    config
)
```

**Bases:** `Module`

Canonical vision MLP.

**`act`** `= ACT2FN[config.hidden_act]`

---

**`c_fc`**

---

**`c_proj`**

---

```python
nemo_automodel.components.models.muse_glimmer.vision.MuseGlimmerVisionMLP.forward(
    x: torch.Tensor
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.muse_glimmer.vision.MuseGlimmerVisionRotaryEmbedding(
    config
)
```

**Bases:** `Module`

Canonical independent-frequency two-axis vision RoPE.

```python
nemo_automodel.components.models.muse_glimmer.vision.MuseGlimmerVisionRotaryEmbedding.forward(
    x: torch.Tensor,
    position_ids: torch.Tensor
) -> tuple[torch.Tensor, torch.Tensor]
```

```python
nemo_automodel.components.models.muse_glimmer.vision.apply_rotary_pos_emb_vision(
    q: torch.Tensor,
    k: torch.Tensor,
    cos: torch.Tensor,
    sin: torch.Tensor
) -> tuple[torch.Tensor, torch.Tensor]
```

Apply the canonical split-half vision RoPE in float32.

```python
nemo_automodel.components.models.muse_glimmer.vision.get_vision_bilinear_indices_and_weights(
    grid_thw: torch.Tensor,
    num_grid_per_side: int,
    spatial_merge_size: int
) -> tuple[torch.Tensor, torch.Tensor]
```

Copy the checkpoint's grid-sample-equivalent position interpolation.

```python
nemo_automodel.components.models.muse_glimmer.vision.get_vision_cu_seqlens(
    grid_thw: torch.Tensor
) -> torch.Tensor
```

Return one packed-attention segment per frame.

```python
nemo_automodel.components.models.muse_glimmer.vision.get_vision_position_ids(
    grid_thw: torch.Tensor,
    spatial_merge_size: int
) -> torch.Tensor
```

Build canonical block-major two-dimensional vision positions.

```python
nemo_automodel.components.models.muse_glimmer.vision.get_vision_window_index(
    grid_thw: torch.Tensor,
    spatial_merge_size: int,
    window_size: int,
    patch_size: int
) -> tuple[torch.Tensor, torch.Tensor]
```

Copy the canonical window-attention permutation and segment lengths.

```python
nemo_automodel.components.models.muse_glimmer.vision.rotate_half(
    x: torch.Tensor
) -> torch.Tensor
```

Rotate the two halves of the hidden dimension.