> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.models.kimi_linear.config

Configuration for Moonshot Kimi Linear checkpoints.

## Module Contents

### Classes

| Name                                                                                              | Description                                                                |
| ------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------- |
| [`KimiLinear48BConfig`](#nemo_automodel-components-models-kimi_linear-config-KimiLinear48BConfig) | HF-compatible configuration for Kimi Linear 48B A3B causal LM checkpoints. |

### API

```python
class nemo_automodel.components.models.kimi_linear.config.KimiLinear48BConfig(
    vocab_size: int = 163840,
    hidden_size: int = 4096,
    head_dim: int | None = None,
    intermediate_size: int = 11008,
    num_hidden_layers: int = 32,
    num_attention_heads: int = 32,
    num_key_value_heads: int | None = None,
    hidden_act: str = 'silu',
    initializer_range: float = 0.02,
    rms_norm_eps: float = 1e-06,
    use_cache: bool = True,
    pad_token_id: int = 0,
    bos_token_id: int = 1,
    eos_token_id: int = 2,
    rope_theta: float = 10000.0,
    rope_scaling: dict[str, typing.Any] | None = None,
    tie_word_embeddings: bool = False,
    moe_intermediate_size: int | None = None,
    moe_renormalize: bool = True,
    moe_router_activation_func: str = 'sigmoid',
    num_experts: int | None = None,
    num_experts_per_token: int | None = None,
    num_shared_experts: int = 0,
    routed_scaling_factor: float = 1.0,
    first_k_dense_replace: int = 0,
    moe_layer_freq: int = 1,
    use_grouped_topk: bool = True,
    num_expert_group: int = 1,
    topk_group: int = 1,
    q_lora_rank: int | None = None,
    kv_lora_rank: int | None = None,
    qk_nope_head_dim: int | None = None,
    qk_rope_head_dim: int | None = None,
    v_head_dim: int | None = None,
    mla_use_nope: bool | None = False,
    num_nextn_predict_layers: int = 0,
    linear_attn_config: dict[str, typing.Any] | None = None,
    kda_mode: str = 'chunk',
    kda_unpad_inputs: bool = True,
    kda_use_fused_gate: bool = True,
    kda_use_qk_l2norm_in_kernel: bool = True,
    kwargs: typing.Any = {}
)
```

**Bases:** `PretrainedConfig`

HF-compatible configuration for Kimi Linear 48B A3B causal LM checkpoints.

Moonshot publishes both this model and the Kimi K3 text backbone under
`model_type: "kimi_linear"` with `architectures: ["KimiLinearForCausalLM"]`,
so neither field tells the two families apart. Automodel gives this one a
distinct identity (`kimi_linear_48b_a3b` / `KimiLinear48BForCausalLM`) and
leaves `kimi_linear` to the K3 text config. A published Moonshot checkpoint
therefore has to name this class explicitly, as the example recipes do; a
checkpoint saved by Automodel already carries the distinct identity.

**`architectures`** `= list(type(self).architectures)`

---

**`head_dim`**

---

**`is_linear_attn`** `bool`

Return whether any decoder layer uses Kimi Delta Attention.

---

**`is_mla`** `bool`

Return whether full-attention layers use Kimi MLA projection fields.

---

**`is_moe`** `bool`

Return whether the checkpoint config declares routed experts.

---

**`keys_to_ignore_at_inference`** `= ['past_key_values']`

---

**`model_type`** `= type(self).model_type`

---

**`num_key_value_heads`**

---

```python
nemo_automodel.components.models.kimi_linear.config.KimiLinear48BConfig.is_kda_layer(
    layer_idx: int
) -> bool
```

Return whether a zero-based layer index is configured as KDA.