nemo_automodel.components.models.glm5_next.config

AutoModel-owned configuration for GLM-5.3-Flash.

The released checkpoint requires Transformers 5.16, while AutoModel’s current runtime baseline predates the upstream glm5_next config. These classes keep the checkpoint field protocol stable and allow AutoConfig to resolve the model without remote code or a dependency bump.

Module Contents

Classes

Name	Description
`Glm5NextConfig`	Top-level GLM-5.3 vision-language configuration.
`Glm5NextTextConfig`	Configuration for the GLM-5.3 hybrid KDA/KPool-DSA text backbone.
`Glm5NextVisionConfig`	Configuration for the GLM-5.3 image/video encoder.

Functions

Name	Description
`_json_safe_value`	Return a JSON-serializable representation of a config value.

API

class nemo_automodel.components.models.glm5_next.config.Glm5NextConfig(
    text_config: dict[str, typing.Any] | nemo_automodel.components.models.glm5_next.config.Glm5NextTextConfig | None = None,
    vision_config: dict[str, typing.Any] | nemo_automodel.components.models.glm5_next.config.Glm5NextVisionConfig | None = None,
    image_token_id: int = 154854,
    video_token_id: int = 154855,
    image_start_token_id: int = 154830,
    image_end_token_id: int = 154831,
    video_start_token_id: int = 154832,
    video_end_token_id: int = 154833,
    tie_word_embeddings: bool = False,
    kwargs: typing.Any = {}
)

Bases: PretrainedConfig

Top-level GLM-5.3 vision-language configuration.

hidden_size

= text_config.hidden_size

keys_to_ignore_at_inference

= ['past_key_values']

max_position_embeddings

= text_config.max_position_embeddings

model_type

= 'glm5_next'

sub_configs

vocab_size

= text_config.vocab_size

nemo_automodel.components.models.glm5_next.config.Glm5NextConfig.get_text_config(
    decoder: bool = False
) -> nemo_automodel.components.models.glm5_next.config.Glm5NextTextConfig

Return the decoder config using the Transformers multimodal protocol.

nemo_automodel.components.models.glm5_next.config.Glm5NextConfig.to_dict() -> dict[str, typing.Any]

class nemo_automodel.components.models.glm5_next.config.Glm5NextTextConfig(
    vocab_size: int = 154880,
    hidden_size: int = 4096,
    intermediate_size: int = 12288,
    moe_intermediate_size: int = 2048,
    num_hidden_layers: int = 45,
    num_attention_heads: int = 64,
    num_key_value_heads: int = 64,
    n_shared_experts: int = 1,
    n_routed_experts: int = 288,
    routed_scaling_factor: float = 2.5,
    kv_lora_rank: int = 512,
    q_lora_rank: int = 1536,
    qk_rope_head_dim: int = 0,
    qk_nope_head_dim: int = 256,
    v_head_dim: int = 256,
    n_group: int = 1,
    topk_group: int = 1,
    num_experts_per_tok: int = 8,
    norm_topk_prob: bool = True,
    mlp_layer_types: list[str] | None = None,
    layer_types: list[str] | None = None,
    indexer_types: list[str] | None = None,
    index_topk_pattern: str | list[str] | None = None,
    index_topk_freq: int = 1,
    index_skip_topk_offset: int = 2,
    index_topk: int = 2048,
    index_head_dim: int = 128,
    index_n_heads: int = 32,
    index_kpool: int = 16,
    index_kpool_always_select_tail: bool = True,
    hidden_act: str = 'silu',
    swiglu_limit: float = 10.0,
    linear_head_dim: int = 128,
    linear_num_heads: int = 64,
    linear_conv_kernel_dim: int = 4,
    linear_lower_bound: float | None = -5.0,
    linear_attn_config: dict[str, typing.Any] | None = None,
    hc_mult: int = 4,
    hc_eps: float = 1e-06,
    hc_sinkhorn_iters: int = 20,
    max_position_embeddings: int = 1048576,
    initializer_range: float = 0.02,
    rms_norm_eps: float = 1e-05,
    use_cache: bool = False,
    attention_bias: bool = False,
    attention_dropout: float = 0.0,
    output_router_logits: bool = False,
    router_aux_loss_coef: float = 0.001,
    num_nextn_predict_layers: int = 1,
    pad_token_id: int | None = 154820,
    bos_token_id: int | None = None,
    eos_token_id: int | list[int] | None = None,
    tie_word_embeddings: bool = False,
    kwargs: typing.Any = {}
)

Bases: PretrainedConfig

Configuration for the GLM-5.3 hybrid KDA/KPool-DSA text backbone.

attribute_map

= {'num_local_experts': 'n_routed_experts'}

indexer_types

= list(indexer_types)

keys_to_ignore_at_inference

= ['past_key_values']

layer_types

= list(layer_types)

linear_attn_config

mlp_layer_types

= list(mlp_layer_types)

model_type

= 'glm5_next_text'

num_local_experts

int

Alias used by Hugging Face expert implementations.

qk_head_dim

= qk_nope_head_dim + qk_rope_head_dim

nemo_automodel.components.models.glm5_next.config.Glm5NextTextConfig.to_dict() -> dict[str, typing.Any]

class nemo_automodel.components.models.glm5_next.config.Glm5NextVisionConfig(
    depth: int = 24,
    hidden_size: int = 1024,
    hidden_act: str = 'silu',
    attention_bias: bool = True,
    attention_dropout: float = 0.0,
    num_heads: int = 16,
    in_channels: int = 3,
    image_size: int = 448,
    patch_size: int = 14,
    rms_norm_eps: float = 1e-05,
    spatial_merge_size: int = 2,
    temporal_patch_size: int = 2,
    out_hidden_size: int = 4096,
    intermediate_size: int = 4096,
    projection_intermediate_size: int = 10240,
    initializer_range: float = 0.02,
    swiglu_limit: float = 10.0,
    kwargs: typing.Any = {}
)

Bases: PretrainedConfig

Configuration for the GLM-5.3 image/video encoder.

model_type

= 'glm5_next_vision'

nemo_automodel.components.models.glm5_next.config.Glm5NextVisionConfig.to_dict() -> dict[str, typing.Any]

nemo_automodel.components.models.glm5_next.config._json_safe_value(
    value: typing.Any
) -> typing.Any

Return a JSON-serializable representation of a config value.