> This page is for version Latest.
> For other versions, use one of these documentation indexes:
> - Nightly (default): https://docs.nvidia.com/nemo/automodel/nightly/llms.txt
> - Latest: https://docs.nvidia.com/nemo/automodel/latest/llms.txt
> - 0.7.0 · 26.10: https://docs.nvidia.com/nemo/automodel/v0.7/llms.txt
> - 0.5.0 · 26.06: https://docs.nvidia.com/nemo/automodel/v0.5/llms.txt
> - 0.4.0 · 26.04: https://docs.nvidia.com/nemo/automodel/v0.4/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.

# nemo_automodel.components.models.ernie5_5.config

Configuration and layer schedule of ERNIE 5.5 causal language models.

`Ernie5_5Config` reads a released `config.json` without remote code. The functions of this module derive the
layer schedule from it and hold the checks that the model applies before it allocates parameters:

* `Ernie5_5ModelPlan.from_config`: the attention and feed-forward kind of every decoder layer, and the attention
  schedule of the MTP block that the released checkpoint carries.
* `decoder_only_config`: a copy of a configuration without the MTP block. The model is built from this form only.
* `validate_architecture`: rejects the settings that select an architecture the model does not implement.
* `main_rope_theta` and `hca_rope_scaling`: the two rotary settings in the form the attention layers take.

## Module Contents

### Classes

| Name                                                                                       | Description                                                                                     |
| ------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------- |
| [`Ernie5_5Config`](#nemo_automodel-components-models-ernie5_5-config-Ernie5_5Config)       | ERNIE 5.5 configuration with the values of the released ERNIE-5.5-Flash checkpoint as defaults. |
| [`Ernie5_5LayerPlan`](#nemo_automodel-components-models-ernie5_5-config-Ernie5_5LayerPlan) | Attention and feed-forward kind of one decoder layer.                                           |
| [`Ernie5_5ModelPlan`](#nemo_automodel-components-models-ernie5_5-config-Ernie5_5ModelPlan) | Decoder layer schedule, and the attention schedule of the MTP block.                            |

### Functions

| Name                                                                                                     | Description                                                                             |
| -------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| [`_is_integer`](#nemo_automodel-components-models-ernie5_5-config-_is_integer)                           | Return whether `value` is an integer and not a boolean.                                 |
| [`decoder_only_config`](#nemo_automodel-components-models-ernie5_5-config-decoder_only_config)           | Return a copy of `config` that describes the decoder without the MTP block.             |
| [`hca_rope_scaling`](#nemo_automodel-components-models-ernie5_5-config-hca_rope_scaling)                 | Return the YaRN settings of the HCA compress rotary embedding.                          |
| [`main_rope_theta`](#nemo_automodel-components-models-ernie5_5-config-main_rope_theta)                   | Return the rotary base of the DSA layers.                                               |
| [`validate_architecture`](#nemo_automodel-components-models-ernie5_5-config-validate_architecture)       | Reject the settings that select an architecture the ERNIE 5.5 model does not implement. |
| [`validate_dsa_rope_layout`](#nemo_automodel-components-models-ernie5_5-config-validate_dsa_rope_layout) | Reject DSA checkpoints whose rotary channels are not stored as adjacent pairs.          |
| [`validate_hca_options`](#nemo_automodel-components-models-ernie5_5-config-validate_hca_options)         | Reject the HCA configuration branches that are not implemented.                         |

### Data

[`HFConfigValue`](#nemo_automodel-components-models-ernie5_5-config-HFConfigValue)

[`_DSA_COMPRESS_RATIO`](#nemo_automodel-components-models-ernie5_5-config-_DSA_COMPRESS_RATIO)

[`_HCA_COMPRESS_RATIO`](#nemo_automodel-components-models-ernie5_5-config-_HCA_COMPRESS_RATIO)

[`_HCA_YARN_KEYS`](#nemo_automodel-components-models-ernie5_5-config-_HCA_YARN_KEYS)

### API

```python
class nemo_automodel.components.models.ernie5_5.config.Ernie5_5Config(
    vocab_size: int = 201216,
    hidden_size: int = 4096,
    intermediate_size: int = 14336,
    num_hidden_layers: int = 43,
    max_position_embeddings: int = 1048576,
    compress_ratios: list[int] | None = None,
    first_k_dense_replace: int = 1,
    moe_layer_freq: int = 1,
    num_nextn_predict_layers: int = 0,
    mtp_shared_last_layer: bool = True,
    num_attention_heads: int = 64,
    num_key_value_heads: int = 1,
    head_dim: int = 512,
    qk_rope_head_dim: int = 64,
    q_lora_rank: int = 1024,
    o_lora_rank: int = 1024,
    o_groups: int = 8,
    sliding_window: int = 128,
    compress_rope_theta: float = 160000.0,
    hca_rope_type: str = 'yarn',
    use_qk_norm: bool = True,
    sigmoid_gate_fusion: bool = False,
    qk_norm_type: str = 'per_head',
    use_gated_attn: bool = True,
    gated_attn_act: str = 'sigmoid',
    gated_attn_use_q_lora: bool = True,
    use_vha_attention: bool = True,
    add_full_attention_sink_bias: bool = True,
    attention_bias: bool = False,
    hybrid_mla_attention: str = 'mqa_dsa',
    mqa_split_kv_b_proj: bool = True,
    use_fast_hadamard: bool = True,
    hybrid_mla_num_attention_heads: int = 64,
    hybrid_mla_q_lora_rank: int = 1024,
    hybrid_mla_kv_lora_rank: int = 512,
    hybrid_mla_qk_nope_head_dim: int = 192,
    hybrid_mla_qk_rope_head_dim: int = 64,
    hybrid_mla_v_head_dim: int = 256,
    index_n_heads: int = 64,
    index_head_dim: int = 128,
    index_topk: int = 2048,
    rope_interleave: bool | None = None,
    mqa_latent_rope_adjacent_pairing: bool | None = None,
    rope_theta: float | None = None,
    rope_scaling: dict[str, nemo_automodel.components.models.ernie5_5.config.HFConfigValue] | None = None,
    rope_parameters: dict[str, nemo_automodel.components.models.ernie5_5.config.HFConfigValue] | None = None,
    rotary_base: float | None = None,
    enable_hyper_connections: bool = True,
    hc_mult: int = 4,
    hc_sinkhorn_iters: int = 20,
    hc_eps: float = 1e-06,
    rms_norm_eps: float = 1e-06,
    hidden_act: str = 'situ',
    activation_situ_beta: float = 4.0,
    activation_situ_linear_beta: float = 25.0,
    n_routed_experts: int = 512,
    n_shared_experts: int = 1,
    num_experts_per_tok: int = 10,
    moe_intermediate_size: int = 2048,
    moe_latent_size: int = 2048,
    routed_scaling_factor: float = 3.0,
    routed_scaling_factor_learnable: bool = True,
    scoring_func: str = 'sigmoid',
    norm_topk_prob: bool = True,
    n_group: int = 1,
    topk_group: int = 1,
    moe_split_feature_routing: bool = True,
    latent_moe_use_norm: bool = True,
    multimax: str = 'lm_head',
    initializer_range: float = 0.02,
    pad_token_id: int | None = None,
    tie_word_embeddings: bool = False,
    use_cache: bool = True,
    kwargs: nemo_automodel.components.models.ernie5_5.config.HFConfigValue = {}
)
```

**Bases:** `PretrainedConfig`

ERNIE 5.5 configuration with the values of the released ERNIE-5.5-Flash checkpoint as defaults.

Five fields do not default to the released value:

* `compress_ratios` has no default. It lists one entry per decoder layer, 128 for an HCA layer and -2 for a
  DSA layer; the released file adds one entry for its MTP layer.
* `num_nextn_predict_layers` defaults to 0 (released: 1). The model does not build the MTP block, so a
  configuration written by hand needs no override. The released `config.json` carries the explicit value 1
  and is rejected until the recipe overrides it with 0: see `decoder_only_config`.
* `rope_interleave` defaults to None (released: true). A configuration with a DSA layer must declare the
  rotary pairing of the checkpoint itself, here or in `mqa_latent_rope_adjacent_pairing`.
* `rotary_base` defaults to None (released: 8000000). The released file repeats the main rotary base in this
  key. When it is set it must equal `rope_theta`; with a default, overriding `rope_theta` alone would be a
  conflict for every configuration.
* `pad_token_id` defaults to None (released: 0). The model rejects input that contains the padding token, so
  a configuration must name it explicitly.

Two fields default to None, which selects the released value: `rope_theta` (8000000, unless
`rope_parameters` carries a base) and `rope_scaling` (the YaRN settings of the HCA layers). The attribute
`rope_theta` reads and writes `rope_parameters["rope_theta"]`.

A configuration may carry the rotary settings twice, in `rope_parameters` and in the released keys
`rope_theta` and `rope_scaling`; the two forms must agree. Fields of the released `config.json` that are
not listed here are kept by `PretrainedConfig`, so loading and saving a checkpoint does not drop them.

**`model_type`** `= 'ernie5_5_moe'`

---

**`rope_theta`** `float | None`

Main rotary base, read from `rope_parameters["rope_theta"]`.

Assigning the attribute writes the same entry, so a configuration override of `rope_theta` reaches the
value that `main_rope_theta` returns.

---

```python
nemo_automodel.components.models.ernie5_5.config.Ernie5_5Config.to_dict() -> dict[str, nemo_automodel.components.models.ernie5_5.config.HFConfigValue]
```

Serialize the configuration with the rotary and dtype keys of the released `config.json`.

Transformers keeps the rotary settings in `rope_parameters` and the dtype in `dtype`. Readers of the
released format look up `rope_theta`, `rope_scaling` and `torch_dtype`, so these keys are written
next to the standardized ones, with the same values.

**Returns:** `dict[str, HFConfigValue]`

The serialized configuration.

```python
class nemo_automodel.components.models.ernie5_5.config.Ernie5_5LayerPlan(
    layer_index: int,
    attention: typing.Literal['hca', 'dsa'],
    feed_forward: typing.Literal['dense', 'moe']
)
```

Dataclass

Attention and feed-forward kind of one decoder layer.

**`attention`** `Literal['hca', 'dsa']`

---

**`feed_forward`** `Literal['dense', 'moe']`

---

**`layer_index`** `int`

---

```python
class nemo_automodel.components.models.ernie5_5.config.Ernie5_5ModelPlan(
    decoder_layers: tuple[nemo_automodel.components.models.ernie5_5.config.Ernie5_5LayerPlan, ...],
    mtp_attention: tuple[typing.Literal['hca', 'dsa'], ...],
    mtp_shared_last_layer: bool
)
```

Dataclass

Decoder layer schedule, and the attention schedule of the MTP block.

**`decoder_layers`** `tuple[Ernie5_5LayerPlan, ...]`

---

**`mtp_attention`** `tuple[Literal['hca', 'dsa'], ...]`

---

**`mtp_shared_last_layer`** `bool`

---

```python
nemo_automodel.components.models.ernie5_5.config.Ernie5_5ModelPlan.from_config(
    config: nemo_automodel.components.models.ernie5_5.config.Ernie5_5Config
) -> nemo_automodel.components.models.ernie5_5.config.Ernie5_5ModelPlan
```

classmethod

Derive the layer schedule, keeping the MTP entries of `compress_ratios` apart from the decoder layers.

**Parameters:**

**`config`** `Ernie5_5Config`

Model configuration. `compress_ratios` holds one entry per decoder layer followed by one
entry per MTP layer.

---

**Returns:** `Ernie5_5ModelPlan`

The attention and feed-forward kind of every decoder layer, the attention kind of every MTP layer and

**Raises:**

* `ValueError`: If a layer count is not an integer in its valid range, `moe_layer_freq` is not 1,
  `compress_ratios` does not have one entry per decoder and MTP layer, an entry is neither 128
  (HCA) nor -2 (DSA), or `mtp_shared_last_layer` is not a boolean.

```python
nemo_automodel.components.models.ernie5_5.config._is_integer(
    value: nemo_automodel.components.models.ernie5_5.config.HFConfigValue
) -> bool
```

Return whether `value` is an integer and not a boolean.

```python
nemo_automodel.components.models.ernie5_5.config.decoder_only_config(
    config: nemo_automodel.components.models.ernie5_5.config.Ernie5_5Config
) -> nemo_automodel.components.models.ernie5_5.config.Ernie5_5Config
```

Return a copy of `config` that describes the decoder without the MTP block.

The model does not build or train the MTP block of the released checkpoint, so it accepts a configuration only
when `num_nextn_predict_layers` is 0. The released `compress_ratios` then still ends with the entry of the
MTP layer, which shares the attention kind of the last decoder layer; the copy drops that entry and clears
`mtp_shared_last_layer`, so that a configuration saved from the model matches the weights saved with it.

**Parameters:**

**`config`** `Ernie5_5Config`

Model configuration. It is not modified.

---

**Returns:** `Ernie5_5Config`

A deep copy with `num_nextn_predict_layers == 0`, one `compress_ratios` entry per decoder layer and

**Raises:**

* `ValueError`: If `num_nextn_predict_layers` is not 0, `compress_ratios` has neither one entry per decoder
  layer nor one additional entry of a shared MTP layer, or `Ernie5_5ModelPlan.from_config` rejects
  the result.

```python
nemo_automodel.components.models.ernie5_5.config.hca_rope_scaling(
    config: nemo_automodel.components.models.ernie5_5.config.Ernie5_5Config
) -> dict[str, nemo_automodel.components.models.ernie5_5.config.HFConfigValue]
```

Return the YaRN settings of the HCA compress rotary embedding.

**Parameters:**

**`config`** `Ernie5_5Config`

Model configuration.

---

**Returns:** `dict[str, HFConfigValue]`

Mapping with `type` set to `"yarn"` and the keys `factor`, `original_max_position_embeddings`,

**Raises:**

* `ValueError`: If `config.rope_parameters` lacks one of the four keys.

```python
nemo_automodel.components.models.ernie5_5.config.main_rope_theta(
    config: nemo_automodel.components.models.ernie5_5.config.Ernie5_5Config
) -> float
```

Return the rotary base of the DSA layers.

**Parameters:**

**`config`** `Ernie5_5Config`

Model configuration. `rope_parameters["rope_theta"]` holds the base; `rotary_base`, when set,
must repeat it.

---

**Returns:** `float`

The positive rotary base.

**Raises:**

* `ValueError`: If the base is missing or not positive, or `rotary_base` differs from it.

```python
nemo_automodel.components.models.ernie5_5.config.validate_architecture(
    config: nemo_automodel.components.models.ernie5_5.config.Ernie5_5Config
) -> None
```

Reject the settings that select an architecture the ERNIE 5.5 model does not implement.

The model implements one variant of the attention, feed-forward and output options of the released
checkpoint. The options it reads are checked individually; the released flags it hard-codes are compared with
the implemented value.

**Parameters:**

**`config`** `Ernie5_5Config`

Model configuration.

---

**Raises:**

* `ValueError`: If the layer schedule is invalid, the activation is not SiTU, the MoE does not have exactly
  one shared expert and one router group with split-feature routing and a latent norm, the mHC and
  RMSNorm epsilons differ, or a rotary setting is missing or inconsistent.
* `NotImplementedError`: If a DSA layer uses the non-adjacent rotary layout, an HCA layer uses an option
  that is not implemented, or a released flag differs from the implemented variant.

```python
nemo_automodel.components.models.ernie5_5.config.validate_dsa_rope_layout(
    config: nemo_automodel.components.models.ernie5_5.config.Ernie5_5Config
) -> None
```

Reject DSA checkpoints whose rotary channels are not stored as adjacent pairs.

`mqa_latent_rope_adjacent_pairing` takes precedence over `rope_interleave` when it is set.

**Parameters:**

**`config`** `Ernie5_5Config`

Model configuration.

---

**Raises:**

* `ValueError`: If neither field holds a boolean.
* `NotImplementedError`: If the checkpoint declares the non-adjacent layout, which would need a permutation of
  the rotary rows that is not implemented.

```python
nemo_automodel.components.models.ernie5_5.config.validate_hca_options(
    config: nemo_automodel.components.models.ernie5_5.config.Ernie5_5Config
) -> None
```

Reject the HCA configuration branches that are not implemented.

**Parameters:**

**`config`** `Ernie5_5Config`

Model configuration.

---

**Raises:**

* `NotImplementedError`: If Q/K normalization is disabled, the HCA rotary type is not YaRN, or the fused
  sigmoid gate is enabled.
* `ValueError`: If the YaRN settings are incomplete.

```python
nemo_automodel.components.models.ernie5_5.config.HFConfigValue: TypeAlias = str | int | float | bool | torch.dtype | list['HFConfigValue'] | dict[str, 'HFCo...
```

```python
nemo_automodel.components.models.ernie5_5.config._DSA_COMPRESS_RATIO = -2
```

```python
nemo_automodel.components.models.ernie5_5.config._HCA_COMPRESS_RATIO = 128
```

```python
nemo_automodel.components.models.ernie5_5.config._HCA_YARN_KEYS = ('factor', 'original_max_position_embeddings', 'beta_fast', 'beta_slow')
```