> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.models.glm5_next.model

Native AutoModel implementation of GLM-5.3-Flash.

## Module Contents

### Classes

| Name                                                                                                                     | Description                                                           |
| ------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------- |
| [`Glm5NextForConditionalGeneration`](#nemo_automodel-components-models-glm5_next-model-Glm5NextForConditionalGeneration) | Trainable GLM-5.3 VLM with EP and contiguous packed CP support.       |
| [`Glm5NextModel`](#nemo_automodel-components-models-glm5_next-model-Glm5NextModel)                                       | Checkpoint-layout container for `visual` and `language_model`.        |
| [`Glm5NextTextModel`](#nemo_automodel-components-models-glm5_next-model-Glm5NextTextModel)                               | Embedding, mHC decoder stack, mean stream collapse and final RMSNorm. |

### Functions

| Name                                                                                                           | Description                                                          |
| -------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------- |
| [`_packed_context_from_inputs`](#nemo_automodel-components-models-glm5_next-model-_packed_context_from_inputs) | Build one global document map for a non-CP forward.                  |
| [`build_glm5_next_moe_config`](#nemo_automodel-components-models-glm5_next-model-build_glm5_next_moe_config)   | Translate the GLM router/expert contract to AutoModel's grouped MoE. |

### Data

[`ModelClass`](#nemo_automodel-components-models-glm5_next-model-ModelClass)

[`__all__`](#nemo_automodel-components-models-glm5_next-model-__all__)

### API

```python
class nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration(
    config: nemo_automodel.components.models.glm5_next.config.Glm5NextConfig,
    moe_config: nemo_automodel.components.moe.config.MoEConfig | None = None,
    backend: nemo_automodel.components.models.common.BackendConfig | None = None,
    kwargs: typing.Any = {}
)
```

**Bases:** [HFCheckpointingMixin](/nemo-automodel/nemo_automodel/components/models/common/hf_checkpointing_mixin#nemo_automodel-components-models-common-hf_checkpointing_mixin-HFCheckpointingMixin), `Module`, [MoEFSDPSyncMixin](/nemo-automodel/nemo_automodel/components/moe/fsdp_mixin#nemo_automodel-components-moe-fsdp_mixin-MoEFSDPSyncMixin)

Trainable GLM-5.3 VLM with EP and contiguous packed CP support.

**`_keep_in_fp32_modules_strict`**

---

**`_packed_cp_attn_backends`** `= ('sdpa', 'cudnn')`

---

**`backend`**

---

**`language_model`** `Glm5NextTextModel`

Expose the text module through the multimodal discovery protocol.

---

**`lm_head`**

---

**`model`**

---

**`state_dict_adapter`**

---

**`tie_word_embeddings_support`** `TieSupport = TieSupport.UNTIED_ONLY`

---

**`vocab_size`** `= text_config.vocab_size`

---

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration._embed_and_splice(
    input_ids: torch.Tensor,
    pixel_values: torch.Tensor | None,
    image_grid_thw: torch.Tensor | None
) -> torch.Tensor
```

Embed the full sequence and replace image placeholder positions.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.forward(
    input_ids: torch.Tensor | None = None,
    attention_mask: torch.Tensor | None = None,
    position_ids: torch.Tensor | None = None,
    inputs_embeds: torch.Tensor | None = None,
    padding_mask: torch.Tensor | None = None,
    pixel_values: torch.Tensor | None = None,
    image_grid_thw: torch.Tensor | None = None,
    pixel_values_videos: torch.Tensor | None = None,
    logits_to_keep: int | torch.Tensor = 0,
    output_hidden_states: bool | None = None,
    kwargs: typing.Any = {}
) -> transformers.modeling_outputs.CausalLMOutputWithPast
```

Run image splice, contiguous CP slicing, text decoding and lm head.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.from_config(
    config: nemo_automodel.components.models.glm5_next.config.Glm5NextConfig,
    moe_config: nemo_automodel.components.moe.config.MoEConfig | None = None,
    backend: nemo_automodel.components.models.common.BackendConfig | None = None,
    kwargs: typing.Any = {}
) -> 'Glm5NextForConditionalGeneration'
```

classmethod

Construct from an already resolved native config.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.from_pretrained(
    pretrained_model_name_or_path: str,
    model_args: typing.Any = (),
    kwargs: typing.Any = {}
) -> 'Glm5NextForConditionalGeneration'
```

classmethod

Resolve the local config; checkpoint loading is owned by AutoModel.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.get_image_features(
    pixel_values: torch.Tensor,
    image_grid_thw: torch.Tensor
) -> nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionOutput
```

Return raw and merged features for flattened image patches.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.get_input_embeddings() -> torch.nn.Module
```

Return the token embedding table.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.get_output_embeddings() -> torch.nn.Module
```

Return the untied language-model head.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.initialize_weights(
    buffer_device: torch.device | None = None,
    dtype: torch.dtype = torch.bfloat16
) -> None
```

Initialize all tensors for checkpoint-free construction.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.prepare_model_inputs_for_cp(
    batch: dict[str, typing.Any],
    num_chunks: int = 1
) -> dict[str, typing.Any]
```

Install GLM's contiguous packed sharder while leaving media and ids global.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.set_input_embeddings(
    value: torch.nn.Module
) -> None
```

Replace the token embedding table.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.set_output_embeddings(
    value: torch.nn.Module
) -> None
```

Replace the language-model head.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.update_moe_gate_bias() -> None
```

Update no-aux-loss router correction biases after an optimizer step.

```python
class nemo_automodel.components.models.glm5_next.model.Glm5NextModel(
    config: nemo_automodel.components.models.glm5_next.config.Glm5NextConfig,
    backend: nemo_automodel.components.models.common.BackendConfig,
    moe_config: nemo_automodel.components.moe.config.MoEConfig | None = None,
    moe_overrides: dict[str, typing.Any] | None = None
)
```

**Bases:** `Module`

Checkpoint-layout container for `visual` and `language_model`.

**`language_model`**

---

**`visual`** `= Glm5NextVisionModel(config.vision_config)`

---

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextModel.get_image_features(
    pixel_values: torch.Tensor,
    image_grid_thw: torch.Tensor
) -> nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionOutput
```

Encode image patches and split-free concatenated features.

```python
class nemo_automodel.components.models.glm5_next.model.Glm5NextTextModel(
    config: nemo_automodel.components.models.glm5_next.config.Glm5NextTextConfig,
    backend: nemo_automodel.components.models.common.BackendConfig,
    moe_config: nemo_automodel.components.moe.config.MoEConfig | None = None,
    moe_overrides: dict[str, typing.Any] | None = None
)
```

**Bases:** `Module`

Embedding, mHC decoder stack, mean stream collapse and final RMSNorm.

**`embed_tokens`**

---

**`layers`**

---

**`moe_config`**

---

**`norm`**

---

**`padding_idx`** `= config.pad_token_id`

---

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextTextModel.forward(
    input_ids: torch.Tensor | None = None,
    inputs_embeds: torch.Tensor | None = None,
    glm5_next_packed_context: nemo_automodel.components.models.glm5_next.cp.Glm5NextPackedContext,
    padding_mask: torch.Tensor | None = None,
    kwargs: typing.Any = {}
) -> torch.Tensor
```

Run `[batch, local_sequence]` ids/embeddings through the text model.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextTextModel.init_weights(
    buffer_device: torch.device
) -> None
```

Initialize a checkpoint-free text model on `buffer_device`.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextTextModel.update_moe_gate_bias() -> None
```

Update every sparse layer's no-aux-loss routing correction bias.

```python
nemo_automodel.components.models.glm5_next.model._packed_context_from_inputs(
    input_ids: torch.Tensor,
    attention_mask: torch.Tensor | None,
    padding_mask: torch.Tensor | None,
    cu_seqlens: torch.Tensor | None,
    packed_seq_ids: torch.Tensor | None
) -> nemo_automodel.components.models.glm5_next.cp.Glm5NextPackedContext
```

Build one global document map for a non-CP forward.

```python
nemo_automodel.components.models.glm5_next.model.build_glm5_next_moe_config(
    config: nemo_automodel.components.models.glm5_next.config.Glm5NextTextConfig,
    dtype: torch.dtype,
    overrides: dict[str, typing.Any] | None = None
) -> nemo_automodel.components.moe.config.MoEConfig
```

Translate the GLM router/expert contract to AutoModel's grouped MoE.

```python
nemo_automodel.components.models.glm5_next.model.ModelClass = Glm5NextForConditionalGeneration
```

```python
nemo_automodel.components.models.glm5_next.model.__all__ = ['Glm5NextForConditionalGeneration', 'Glm5NextModel', 'Glm5NextTextModel', 'buil...
```