> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.models.glm5_next.vision

GLM-5.3 image encoder with checkpoint-compatible module names.

## Module Contents

### Classes

| Name                                                                                                                | Description                                                         |
| ------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
| [`Glm5NextVisionAttention`](#nemo_automodel-components-models-glm5_next-vision-Glm5NextVisionAttention)             | Bidirectional per-image attention over `[vision_tokens, hidden]`.   |
| [`Glm5NextVisionBlock`](#nemo_automodel-components-models-glm5_next-vision-Glm5NextVisionBlock)                     | Pre-norm bidirectional vision transformer block.                    |
| [`Glm5NextVisionMLP`](#nemo_automodel-components-models-glm5_next-vision-Glm5NextVisionMLP)                         | Clamped SwiGLU vision feed-forward block.                           |
| [`Glm5NextVisionModel`](#nemo_automodel-components-models-glm5_next-vision-Glm5NextVisionModel)                     | Patch encoder returning image features in the language hidden size. |
| [`Glm5NextVisionOutput`](#nemo_automodel-components-models-glm5_next-vision-Glm5NextVisionOutput)                   | Raw and merged vision token representations.                        |
| [`Glm5NextVisionPatchEmbed`](#nemo_automodel-components-models-glm5_next-vision-Glm5NextVisionPatchEmbed)           | Conv3d patch projection from `[patches, C*T*P*P]` to vision hidden. |
| [`Glm5NextVisionPatchMerger`](#nemo_automodel-components-models-glm5_next-vision-Glm5NextVisionPatchMerger)         | Post-downsample projection in the text hidden dimension.            |
| [`Glm5NextVisionRotaryEmbedding`](#nemo_automodel-components-models-glm5_next-vision-Glm5NextVisionRotaryEmbedding) | Two-axis rotary frequencies used by the vision transformer.         |

### Functions

| Name                                                                                              | Description                                             |
| ------------------------------------------------------------------------------------------------- | ------------------------------------------------------- |
| [`_rotate_half`](#nemo_automodel-components-models-glm5_next-vision-_rotate_half)                 | -                                                       |
| [`_vision_cu_seqlens`](#nemo_automodel-components-models-glm5_next-vision-_vision_cu_seqlens)     | Return per-frame attention boundaries `[segments + 1]`. |
| [`_vision_position_ids`](#nemo_automodel-components-models-glm5_next-vision-_vision_position_ids) | Return block-major H/W positions `[vision_tokens, 2]`.  |

### Data

[`__all__`](#nemo_automodel-components-models-glm5_next-vision-__all__)

### API

```python
class nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionAttention(
    config: nemo_automodel.components.models.glm5_next.config.Glm5NextVisionConfig,
    dtype: torch.dtype
)
```

**Bases:** `Module`

Bidirectional per-image attention over `[vision_tokens, hidden]`.

**`dropout`** `= config.attention_dropout`

---

**`head_dim`** `= config.hidden_size // config.num_heads`

---

**`k_norm`**

---

**`num_heads`** `= config.num_heads`

---

**`proj`**

---

**`q_norm`**

---

**`qkv`**

---

**`scaling`** `= self.head_dim ** -0.5`

---

```python
nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionAttention.forward(
    hidden_states: torch.Tensor,
    cu_seqlens: torch.Tensor,
    position_embeddings: tuple[torch.Tensor, torch.Tensor]
) -> torch.Tensor
```

Attend within each cu-seqlens segment and return `[tokens, hidden]`.

```python
class nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionBlock(
    config: nemo_automodel.components.models.glm5_next.config.Glm5NextVisionConfig,
    dtype: torch.dtype
)
```

**Bases:** `Module`

Pre-norm bidirectional vision transformer block.

**`attn`** `= Glm5NextVisionAttention(config, dtype)`

---

**`mlp`** `= Glm5NextVisionMLP(config, dtype)`

---

**`norm1`**

---

**`norm2`**

---

```python
nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionBlock.forward(
    hidden_states: torch.Tensor,
    cu_seqlens: torch.Tensor,
    position_embeddings: tuple[torch.Tensor, torch.Tensor]
) -> torch.Tensor
```

Transform `[vision_tokens, hidden]` without cross-image attention.

```python
class nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionMLP(
    config: nemo_automodel.components.models.glm5_next.config.Glm5NextVisionConfig,
    dtype: torch.dtype
)
```

**Bases:** `Module`

Clamped SwiGLU vision feed-forward block.

**`down_proj`**

---

**`gate_proj`**

---

**`swiglu_limit`** `= config.swiglu_limit`

---

**`up_proj`**

---

```python
nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionMLP.forward(
    hidden_states: torch.Tensor
) -> torch.Tensor
```

Transform `[tokens, vision_hidden]` with clamped SwiGLU.

```python
class nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionModel(
    config: nemo_automodel.components.models.glm5_next.config.Glm5NextVisionConfig
)
```

**Bases:** `Module`

Patch encoder returning image features in the language hidden size.

**`blocks`**

---

**`downsample`**

---

**`dtype`** `dtype`

Return the patch embedding dtype.

---

**`merger`** `= Glm5NextVisionPatchMerger(config, dtype)`

---

**`patch_embed`** `= Glm5NextVisionPatchEmbed(config, dtype)`

---

**`post_layernorm`**

---

**`rotary_pos_emb`** `= Glm5NextVisionRotaryEmbedding(head_dim // 2)`

---

**`spatial_merge_size`** `= config.spatial_merge_size`

---

```python
nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionModel.forward(
    pixel_values: torch.Tensor,
    grid_thw: torch.Tensor
) -> nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionOutput
```

Encode flattened patches using grid metadata `[images, (t,h,w)]`.

```python
nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionModel.init_weights(
    buffer_device: torch.device,
    init_std: float
) -> None
```

Initialize vision parameters without materializing outside `buffer_device`.

```python
class nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionOutput(
    last_hidden_state: torch.Tensor,
    pooler_output: torch.Tensor
)
```

Dataclass

Raw and merged vision token representations.

**`last_hidden_state`** `Tensor`

---

**`pooler_output`** `Tensor`

---

```python
class nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionPatchEmbed(
    config: nemo_automodel.components.models.glm5_next.config.Glm5NextVisionConfig,
    dtype: torch.dtype
)
```

**Bases:** `Module`

Conv3d patch projection from `[patches, C*T*P*P]` to vision hidden.

**`in_channels`** `= config.in_channels`

---

**`patch_size`** `= config.patch_size`

---

**`proj`**

---

**`temporal_patch_size`** `= config.temporal_patch_size`

---

```python
nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionPatchEmbed.forward(
    pixel_values: torch.Tensor
) -> torch.Tensor
```

Project flattened patches to `[vision_tokens, vision_hidden]`.

```python
class nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionPatchMerger(
    config: nemo_automodel.components.models.glm5_next.config.Glm5NextVisionConfig,
    dtype: torch.dtype
)
```

**Bases:** `Module`

Post-downsample projection in the text hidden dimension.

**`down_proj`** `= nn.Linear(context, dim, bias=False, dtype=dtype)`

---

**`gate_proj`** `= nn.Linear(dim, context, bias=False, dtype=dtype)`

---

**`post_projection_norm`** `= nn.LayerNorm(dim, dtype=dtype)`

---

**`proj`** `= nn.Linear(dim, dim, bias=False, dtype=dtype)`

---

**`swiglu_limit`** `= config.swiglu_limit`

---

**`up_proj`** `= nn.Linear(dim, context, bias=False, dtype=dtype)`

---

```python
nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionPatchMerger.forward(
    hidden_states: torch.Tensor
) -> torch.Tensor
```

Project downsampled image tokens `[tokens, text_hidden]`.

```python
class nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionRotaryEmbedding(
    dim: int,
    theta: float = 10000.0
)
```

**Bases:** `Module`

Two-axis rotary frequencies used by the vision transformer.

```python
nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionRotaryEmbedding.forward(
    position_ids: torch.Tensor
) -> torch.Tensor
```

Map H/W ids `[tokens, 2]` to frequencies `[tokens, head_dim/2]`.

```python
nemo_automodel.components.models.glm5_next.vision._rotate_half(
    hidden_states: torch.Tensor
) -> torch.Tensor
```

```python
nemo_automodel.components.models.glm5_next.vision._vision_cu_seqlens(
    grid_thw: torch.Tensor
) -> torch.Tensor
```

Return per-frame attention boundaries `[segments + 1]`.

```python
nemo_automodel.components.models.glm5_next.vision._vision_position_ids(
    grid_thw: torch.Tensor,
    merge_size: int
) -> torch.Tensor
```

Return block-major H/W positions `[vision_tokens, 2]`.

```python
nemo_automodel.components.models.glm5_next.vision.__all__ = ['Glm5NextVisionModel', 'Glm5NextVisionOutput']
```