> This page is for version 0.5.0 · 26.06.
> For other versions, use one of these documentation indexes:
> - Nightly (default): https://docs.nvidia.com/nemo/automodel/nightly/llms.txt
> - Latest: https://docs.nvidia.com/nemo/automodel/latest/llms.txt
> - 0.7.0 · 26.10: https://docs.nvidia.com/nemo/automodel/v0.7/llms.txt
> - 0.5.0 · 26.06: https://docs.nvidia.com/nemo/automodel/v0.5/llms.txt
> - 0.4.0 · 26.04: https://docs.nvidia.com/nemo/automodel/v0.4/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.models.flint.config

Local configuration for Flint's hybrid-attention MoE architecture.

## Module Contents

### Classes

| Name                                                                        | Description                                                 |
| --------------------------------------------------------------------------- | ----------------------------------------------------------- |
| [`FlintConfig`](#nemo_automodel-components-models-flint-config-FlintConfig) | Architecture settings matching the source Flint checkpoint. |

### API

```python
class nemo_automodel.components.models.flint.config.FlintConfig()
```

**Bases:** `PreTrainedConfig`

Architecture settings matching the source Flint checkpoint.

The local class removes dependence on checkpoint remote code. Full layers
have no RoPE; sliding layers rotate all head channels with NeoX RoPE. The
window counts the current token. `router_aux_loss_coef` is preserved as
checkpoint metadata; the native training implementation requires an explicit
zero value until the original auxiliary objective is specified.

`attention_compute_dtype='fp16'` is an explicit TE-only QKV/kernel-output
boundary within otherwise FP32 projection compute. It does not change the
bulk parameter dtype or enable autocast. Small-gradient training requires an
appropriate loss-scaling policy; the default None retains native precision.

**`attention_bias`** `bool = False`

---

**`attention_compute_dtype`** `str | None = None`

---

**`attention_dropout`** `float | int = 0.0`

---

**`bos_token_id`** `int | None = None`

---

**`decoder_sparse_step`** `int = 1`

---

**`eos_token_id`** `int | list[int] | None = None`

---

**`head_dim`** `int = 128`

---

**`hidden_act`** `str = 'silu'`

---

**`hidden_size`** `int = 2048`

---

**`initializer_range`** `float = 0.02`

---

**`intermediate_size`** `int = 6144`

---

**`keys_to_ignore_at_inference`** `= ['past_key_values']`

---

**`layer_types`** `list[str] | None = None`

---

**`max_position_embeddings`** `int = 262144`

---

**`mlp_only_layers`** `list[int] | None = None`

---

**`model_type`** `= 'flint'`

---

**`moe_intermediate_size`** `int = 512`

---

**`no_rope_layer_indices`** `list[int] | None = None`

---

**`norm_topk_prob`** `bool = True`

---

**`num_attention_heads`** `int = 32`

---

**`num_experts`** `int = 256`

---

**`num_experts_per_tok`** `int = 6`

---

**`num_hidden_layers`** `int = 49`

---

**`num_key_value_heads`** `int = 4`

---

**`output_router_logits`** `bool = False`

---

**`pad_token_id`** `int | None = None`

---

**`residual_dtype`** `str | None = None`

---

**`rms_norm_eps`** `float = 1e-06`

---

**`rope_parameters`** `RopeParameters | dict[str, object] | None = None`

---

**`router_aux_loss_coef`** `float = 0.0`

---

**`router_score_function`** `str = 'sigmoid'`

---

**`schedule_num_hidden_layers`** `int | None = None`

---

**`shared_expert_intermediate_size`** `int = 1024`

---

**`sliding_window`** `int | None = 513`

---

**`softmax_type`** `str = 'learnable'`

---

**`tie_word_embeddings`** `bool = False`

---

**`use_cache`** `bool = False`

---

**`use_sliding_window`** `bool = True`

---

**`vocab_size`** `int = 150144`

---

```python
nemo_automodel.components.models.flint.config.FlintConfig.__post_init__(
    kwargs: object = {}
) -> None
```

Resolve schedules and upstream RoPE fields, then validate the contract.

```python
nemo_automodel.components.models.flint.config.FlintConfig.build(
    backend: nemo_automodel.components.models.common.BackendConfig | None = None
) -> nemo_automodel.components.models.flint.model.FlintForCausalLM
```

Construct a native model without changing these declarative settings.

**Parameters:**

**`backend`** `BackendConfig | None` — default: None

Runtime kernel choices; defaults to the verified eager/torch path.

---

**Returns:** `FlintForCausalLM`

An initialized native causal language model.

```python
nemo_automodel.components.models.flint.config.FlintConfig.validate_layer_type() -> None
```

Validate the preserved source schedule independently of active prefix depth.