> This page is for version 0.5.0 · 26.06.
> For other versions, use one of these documentation indexes:
> - Nightly (default): https://docs.nvidia.com/nemo/automodel/nightly/llms.txt
> - Latest: https://docs.nvidia.com/nemo/automodel/latest/llms.txt
> - 0.5.0 · 26.06: https://docs.nvidia.com/nemo/automodel/v0.5/llms.txt
> - 0.4.0 · 26.04: https://docs.nvidia.com/nemo/automodel/v0.4/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.models.glm5_next

Native AutoModel support for `zai-org/GLM-5.3-Flash`.

## Submodules

* **[`nemo_automodel.components.models.glm5_next.config`](/nemo-automodel/nemo_automodel/components/models/glm5_next/config)**
* **[`nemo_automodel.components.models.glm5_next.cp`](/nemo-automodel/nemo_automodel/components/models/glm5_next/cp)**
* **[`nemo_automodel.components.models.glm5_next.image_processing`](/nemo-automodel/nemo_automodel/components/models/glm5_next/image_processing)**
* **[`nemo_automodel.components.models.glm5_next.layers`](/nemo-automodel/nemo_automodel/components/models/glm5_next/layers)**
* **[`nemo_automodel.components.models.glm5_next.model`](/nemo-automodel/nemo_automodel/components/models/glm5_next/model)**
* **[`nemo_automodel.components.models.glm5_next.processing`](/nemo-automodel/nemo_automodel/components/models/glm5_next/processing)**
* **[`nemo_automodel.components.models.glm5_next.state_dict_adapter`](/nemo-automodel/nemo_automodel/components/models/glm5_next/state_dict_adapter)**
* **[`nemo_automodel.components.models.glm5_next.vision`](/nemo-automodel/nemo_automodel/components/models/glm5_next/vision)**

## Package Contents

### Classes

| Name                                                                                                                     | Description                                                       |
| ------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------- |
| [`Glm5NextConfig`](#nemo_automodel-components-models-glm5_next-config-Glm5NextConfig)                                    | Top-level GLM-5.3 vision-language configuration.                  |
| [`Glm5NextForConditionalGeneration`](#nemo_automodel-components-models-glm5_next-model-Glm5NextForConditionalGeneration) | Trainable GLM-5.3 VLM with EP and contiguous packed CP support.   |
| [`Glm5NextTextConfig`](#nemo_automodel-components-models-glm5_next-config-Glm5NextTextConfig)                            | Configuration for the GLM-5.3 hybrid KDA/KPool-DSA text backbone. |
| [`Glm5NextVisionConfig`](#nemo_automodel-components-models-glm5_next-config-Glm5NextVisionConfig)                        | Configuration for the GLM-5.3 image/video encoder.                |

### Data

[`ModelClass`](#nemo_automodel-components-models-glm5_next-ModelClass)

### API

```python
class nemo_automodel.components.models.glm5_next.config.Glm5NextConfig(
    text_config: dict[str, typing.Any] | nemo_automodel.components.models.glm5_next.config.Glm5NextTextConfig | None = None,
    vision_config: dict[str, typing.Any] | nemo_automodel.components.models.glm5_next.config.Glm5NextVisionConfig | None = None,
    image_token_id: int = 154854,
    video_token_id: int = 154855,
    image_start_token_id: int = 154830,
    image_end_token_id: int = 154831,
    video_start_token_id: int = 154832,
    video_end_token_id: int = 154833,
    tie_word_embeddings: bool = False,
    kwargs: typing.Any = {}
)
```

**Bases:** `PretrainedConfig`

Top-level GLM-5.3 vision-language configuration.

**`hidden_size`** `= text_config.hidden_size`

---

**`keys_to_ignore_at_inference`** `= ['past_key_values']`

---

**`max_position_embeddings`** `= text_config.max_position_embeddings`

---

**`model_type`** `= 'glm5_next'`

---

**`sub_configs`**

---

**`vocab_size`** `= text_config.vocab_size`

---

```python
nemo_automodel.components.models.glm5_next.config.Glm5NextConfig.get_text_config(
    decoder: bool = False
) -> nemo_automodel.components.models.glm5_next.config.Glm5NextTextConfig
```

Return the decoder config using the Transformers multimodal protocol.

```python
nemo_automodel.components.models.glm5_next.config.Glm5NextConfig.to_dict() -> dict[str, typing.Any]
```

```python
class nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration(
    config: nemo_automodel.components.models.glm5_next.config.Glm5NextConfig,
    moe_config: nemo_automodel.components.moe.config.MoEConfig | None = None,
    backend: nemo_automodel.components.models.common.BackendConfig | None = None,
    kwargs: typing.Any = {}
)
```

**Bases:** [HFCheckpointingMixin](/nemo-automodel/nemo_automodel/components/models/common/hf_checkpointing_mixin#nemo_automodel-components-models-common-hf_checkpointing_mixin-HFCheckpointingMixin), `Module`, [MoEFSDPSyncMixin](/nemo-automodel/nemo_automodel/components/moe/fsdp_mixin#nemo_automodel-components-moe-fsdp_mixin-MoEFSDPSyncMixin)

Trainable GLM-5.3 VLM with EP and contiguous packed CP support.

**`_keep_in_fp32_modules_strict`**

---

**`_packed_cp_attn_backends`** `= ('sdpa', 'cudnn')`

---

**`backend`**

---

**`language_model`** `Glm5NextTextModel`

Expose the text module through the multimodal discovery protocol.

---

**`lm_head`**

---

**`model`**

---

**`state_dict_adapter`**

---

**`tie_word_embeddings_support`** `TieSupport = TieSupport.UNTIED_ONLY`

---

**`vocab_size`** `= text_config.vocab_size`

---

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration._embed_and_splice(
    input_ids: torch.Tensor,
    pixel_values: torch.Tensor | None,
    image_grid_thw: torch.Tensor | None
) -> torch.Tensor
```

Embed the full sequence and replace image placeholder positions.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.forward(
    input_ids: torch.Tensor | None = None,
    attention_mask: torch.Tensor | None = None,
    position_ids: torch.Tensor | None = None,
    inputs_embeds: torch.Tensor | None = None,
    padding_mask: torch.Tensor | None = None,
    pixel_values: torch.Tensor | None = None,
    image_grid_thw: torch.Tensor | None = None,
    pixel_values_videos: torch.Tensor | None = None,
    logits_to_keep: int | torch.Tensor = 0,
    output_hidden_states: bool | None = None,
    kwargs: typing.Any = {}
) -> transformers.modeling_outputs.CausalLMOutputWithPast
```

Run image splice, contiguous CP slicing, text decoding and lm head.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.from_config(
    config: nemo_automodel.components.models.glm5_next.config.Glm5NextConfig,
    moe_config: nemo_automodel.components.moe.config.MoEConfig | None = None,
    backend: nemo_automodel.components.models.common.BackendConfig | None = None,
    kwargs: typing.Any = {}
) -> 'Glm5NextForConditionalGeneration'
```

classmethod

Construct from an already resolved native config.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.from_pretrained(
    pretrained_model_name_or_path: str,
    model_args: typing.Any = (),
    kwargs: typing.Any = {}
) -> 'Glm5NextForConditionalGeneration'
```

classmethod

Resolve the local config; checkpoint loading is owned by AutoModel.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.get_image_features(
    pixel_values: torch.Tensor,
    image_grid_thw: torch.Tensor
) -> nemo_automodel.components.models.glm5_next.vision.Glm5NextVisionOutput
```

Return raw and merged features for flattened image patches.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.get_input_embeddings() -> torch.nn.Module
```

Return the token embedding table.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.get_output_embeddings() -> torch.nn.Module
```

Return the untied language-model head.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.initialize_weights(
    buffer_device: torch.device | None = None,
    dtype: torch.dtype = torch.bfloat16
) -> None
```

Initialize all tensors for checkpoint-free construction.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.prepare_model_inputs_for_cp(
    batch: dict[str, typing.Any],
    num_chunks: int = 1
) -> dict[str, typing.Any]
```

Install GLM's contiguous packed sharder while leaving media and ids global.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.set_input_embeddings(
    value: torch.nn.Module
) -> None
```

Replace the token embedding table.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.set_output_embeddings(
    value: torch.nn.Module
) -> None
```

Replace the language-model head.

```python
nemo_automodel.components.models.glm5_next.model.Glm5NextForConditionalGeneration.update_moe_gate_bias() -> None
```

Update no-aux-loss router correction biases after an optimizer step.

```python
class nemo_automodel.components.models.glm5_next.config.Glm5NextTextConfig(
    vocab_size: int = 154880,
    hidden_size: int = 4096,
    intermediate_size: int = 12288,
    moe_intermediate_size: int = 2048,
    num_hidden_layers: int = 45,
    num_attention_heads: int = 64,
    num_key_value_heads: int = 64,
    n_shared_experts: int = 1,
    n_routed_experts: int = 288,
    routed_scaling_factor: float = 2.5,
    kv_lora_rank: int = 512,
    q_lora_rank: int = 1536,
    qk_rope_head_dim: int = 0,
    qk_nope_head_dim: int = 256,
    v_head_dim: int = 256,
    n_group: int = 1,
    topk_group: int = 1,
    num_experts_per_tok: int = 8,
    norm_topk_prob: bool = True,
    mlp_layer_types: list[str] | None = None,
    layer_types: list[str] | None = None,
    indexer_types: list[str] | None = None,
    index_topk_pattern: str | list[str] | None = None,
    index_topk_freq: int = 1,
    index_skip_topk_offset: int = 2,
    index_topk: int = 2048,
    index_head_dim: int = 128,
    index_n_heads: int = 32,
    index_kpool: int = 16,
    index_kpool_always_select_tail: bool = True,
    hidden_act: str = 'silu',
    swiglu_limit: float = 10.0,
    linear_head_dim: int = 128,
    linear_num_heads: int = 64,
    linear_conv_kernel_dim: int = 4,
    linear_lower_bound: float | None = -5.0,
    linear_attn_config: dict[str, typing.Any] | None = None,
    hc_mult: int = 4,
    hc_eps: float = 1e-06,
    hc_sinkhorn_iters: int = 20,
    max_position_embeddings: int = 1048576,
    initializer_range: float = 0.02,
    rms_norm_eps: float = 1e-05,
    use_cache: bool = False,
    attention_bias: bool = False,
    attention_dropout: float = 0.0,
    output_router_logits: bool = False,
    router_aux_loss_coef: float = 0.001,
    num_nextn_predict_layers: int = 1,
    pad_token_id: int | None = 154820,
    bos_token_id: int | None = None,
    eos_token_id: int | list[int] | None = None,
    tie_word_embeddings: bool = False,
    kwargs: typing.Any = {}
)
```

**Bases:** `PretrainedConfig`

Configuration for the GLM-5.3 hybrid KDA/KPool-DSA text backbone.

**`attribute_map`** `= {'num_local_experts': 'n_routed_experts'}`

---

**`indexer_types`** `= list(indexer_types)`

---

**`keys_to_ignore_at_inference`** `= ['past_key_values']`

---

**`layer_types`** `= list(layer_types)`

---

**`linear_attn_config`**

---

**`mlp_layer_types`** `= list(mlp_layer_types)`

---

**`model_type`** `= 'glm5_next_text'`

---

**`num_local_experts`** `int`

Alias used by Hugging Face expert implementations.

---

**`qk_head_dim`** `= qk_nope_head_dim + qk_rope_head_dim`

---

```python
nemo_automodel.components.models.glm5_next.config.Glm5NextTextConfig.to_dict() -> dict[str, typing.Any]
```

```python
class nemo_automodel.components.models.glm5_next.config.Glm5NextVisionConfig(
    depth: int = 24,
    hidden_size: int = 1024,
    hidden_act: str = 'silu',
    attention_bias: bool = True,
    attention_dropout: float = 0.0,
    num_heads: int = 16,
    in_channels: int = 3,
    image_size: int = 448,
    patch_size: int = 14,
    rms_norm_eps: float = 1e-05,
    spatial_merge_size: int = 2,
    temporal_patch_size: int = 2,
    out_hidden_size: int = 4096,
    intermediate_size: int = 4096,
    projection_intermediate_size: int = 10240,
    initializer_range: float = 0.02,
    swiglu_limit: float = 10.0,
    kwargs: typing.Any = {}
)
```

**Bases:** `PretrainedConfig`

Configuration for the GLM-5.3 image/video encoder.

**`model_type`** `= 'glm5_next_vision'`

---

```python
nemo_automodel.components.models.glm5_next.config.Glm5NextVisionConfig.to_dict() -> dict[str, typing.Any]
```

```python
nemo_automodel.components.models.glm5_next.ModelClass = Glm5NextForConditionalGeneration
```