> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.models.deepseek_v41.config

Hugging Face checkpoint configuration for DeepSeek-V4.1-Flash.

## Module Contents

### Classes

| Name                                                                                                                   | Description                                                                 |
| ---------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------- |
| [`DeepseekV41Config`](#nemo_automodel-components-models-deepseek_v41-config-DeepseekV41Config)                         | Nested checkpoint configuration for `DeepseekV41ForCausalLM`.               |
| [`DeepseekV41DSparkConfig`](#nemo_automodel-components-models-deepseek_v41-config-DeepseekV41DSparkConfig)             | Validated declarative configuration for the released V4.1 DSpark module.    |
| [`DeepseekV41DSparkTargetConfig`](#nemo_automodel-components-models-deepseek_v41-config-DeepseekV41DSparkTargetConfig) | Construction settings for the frozen, text-only V4.1 DSpark target.         |
| [`DeepseekV41TextConfig`](#nemo_automodel-components-models-deepseek_v41-config-DeepseekV41TextConfig)                 | Declarative configuration of the CSA2, single-pass mHC and Engram backbone. |
| [`DeepseekV41VisionConfig`](#nemo_automodel-components-models-deepseek_v41-config-DeepseekV41VisionConfig)             | Configuration of the released 2D-RoPE vision encoder and image sizing.      |
| [`_DSparkDraftOptions`](#nemo_automodel-components-models-deepseek_v41-config-_DSparkDraftOptions)                     | Declarative DSpark settings supplied by the generic training recipe.        |

### API

```python
class nemo_automodel.components.models.deepseek_v41.config.DeepseekV41Config(
    text_config: dict[str, typing.Any] | nemo_automodel.components.models.deepseek_v41.config.DeepseekV41TextConfig | None = None,
    vision_config: dict[str, typing.Any] | nemo_automodel.components.models.deepseek_v41.config.DeepseekV41VisionConfig | None = None,
    image_token_id: int = 129264,
    dtype: str = 'bfloat16',
    pad_token_id: int | None = 2,
    bos_token_id: int = 0,
    eos_token_id: int = 1,
    tie_word_embeddings: bool = False,
    architectures: list[str] | None = None,
    kwargs: typing.Any = {}
)
```

**Bases:** `PretrainedConfig`

Nested checkpoint configuration for `DeepseekV41ForCausalLM`.

Hugging Face dictionaries are materialized at this boundary; model
components receive the typed `text_config` and `vision_config`.
Quantization metadata, when present in a checkpoint, is preserved by the
base class rather than imposed on checkpoint-free BF16 configurations.

**`keys_to_ignore_at_inference`** `= ['past_key_values']`

---

**`model_type`** `= 'deepseek_v41'`

---

**`sub_configs`**

---

```python
nemo_automodel.components.models.deepseek_v41.config.DeepseekV41Config.build_dspark_draft(
    options: nemo_automodel.components.models.deepseek_v41.config._DSparkDraftOptions
) -> nemo_automodel.components.models.deepseek_v41.dspark.DeepseekV41DSparkModel
```

Build the model-owned DSpark training adapter.

**Parameters:**

**`options`** `_DSparkDraftOptions`

Declarative draft settings supplied by the DSpark recipe.

---

**Returns:** `DeepseekV41DSparkModel`

Native DeepSeek V4.1 DSpark model ready for device placement.

```python
nemo_automodel.components.models.deepseek_v41.config.DeepseekV41Config.build_tokenizer() -> transformers.PreTrainedTokenizerFast
```

Load the checkpoint's fast tokenizer for deterministic Engram hashing.

**Returns:** `PreTrainedTokenizerFast`

The original fast tokenizer from this configuration's checkpoint

**Raises:**

* `ValueError`: The configuration has no checkpoint source. Callers
  constructing a tiny model can supply a tokenizer directly.
* `TypeError`: The checkpoint resolves to a slow tokenizer.

```python
class nemo_automodel.components.models.deepseek_v41.config.DeepseekV41DSparkConfig(
    text_config: nemo_automodel.components.models.deepseek_v41.config.DeepseekV41TextConfig,
    num_draft_layers: int,
    target_layer_ids: tuple[int, ...],
    block_size: int,
    num_anchors: int,
    mask_token_id: int,
    markov_rank: int,
    markov_head_type: str,
    confidence_head_alpha: float,
    confidence_head_with_markov: bool,
    confidence_head_stop_gradient: bool = False,
    quantization_config: dict[str, typing.Any] | None = None
)
```

Dataclass

Validated declarative configuration for the released V4.1 DSpark module.

**`block_size`** `int`

---

**`confidence_head_alpha`** `float`

---

**`confidence_head_stop_gradient`** `bool = False`

---

**`confidence_head_with_markov`** `bool`

---

**`markov_head_type`** `str`

---

**`markov_rank`** `int`

---

**`mask_token_id`** `int`

---

**`num_anchors`** `int`

---

**`num_draft_layers`** `int`

---

**`quantization_config`** `dict[str, Any] | None = None`

---

**`target_layer_ids`** `tuple[int, ...]`

---

**`text_config`** `DeepseekV41TextConfig`

---

```python
nemo_automodel.components.models.deepseek_v41.config.DeepseekV41DSparkConfig.build() -> nemo_automodel.components.models.deepseek_v41.dspark.DeepseekV41DSparkModel
```

Validate the released contract and build its training adapter.

**Returns:** `DeepseekV41DSparkModel`

Newly initialized DeepSeek V4.1 DSpark training model.

**Raises:**

* `ValueError`: If recipe settings disagree with the released checkpoint.

```python
class nemo_automodel.components.models.deepseek_v41.config.DeepseekV41DSparkTargetConfig(
    target_path: str,
    trust_remote_code: bool = False,
    target_num_hidden_layers: int | None = None,
    attn_backend: str = 'tilelang',
    dispatcher: str = 'hybridep',
    experts: str = 'torch_mm',
    enable_fsdp_optimizations: bool = True
)
```

Dataclass

Construction settings for the frozen, text-only V4.1 DSpark target.

The released feature contract requires the full target depth. The native
MTP tensors belong to the separately trained draft.

**`attn_backend`** `str = 'tilelang'`

---

**`dispatcher`** `str = 'hybridep'`

---

**`enable_fsdp_optimizations`** `bool = True`

---

**`experts`** `str = 'torch_mm'`

---

**`target_num_hidden_layers`** `int | None = None`

---

**`target_path`** `str`

---

**`trust_remote_code`** `bool = False`

---

```python
nemo_automodel.components.models.deepseek_v41.config.DeepseekV41DSparkTargetConfig.build(
    device: torch.device,
    compute_dtype: torch.dtype,
    distributed_setup: nemo_automodel.components.distributed.config.DistributedSetup
) -> nemo_automodel.components.models.deepseek_v41.model.DeepseekV41ForCausalLM
```

Load the text target through the supplied EP/FSDP infrastructure.

**Parameters:**

**`device`** `torch.device`

Resolved execution device; the sharded target requires CUDA.

---

**`compute_dtype`** `torch.dtype`

Precision used to load and compute the frozen target.

---

**`distributed_setup`** `DistributedSetup`

Runtime parallelism configuration composed by the recipe.

---

**Returns:** `DeepseekV41ForCausalLM`

The pretrained target with its vision tower disabled.

```python
class nemo_automodel.components.models.deepseek_v41.config.DeepseekV41TextConfig(
    vocab_size: int = 129280,
    hidden_size: int = 5120,
    moe_intermediate_size: int = 2304,
    num_hidden_layers: int = 40,
    num_attention_heads: int = 64,
    num_key_value_heads: int = 1,
    head_dim: int = 512,
    qk_rope_head_dim: int = 64,
    q_lora_rank: int = 1280,
    o_lora_rank: int = 1024,
    o_groups: int = 8,
    hidden_act: str = 'silu',
    swiglu_limit: float = 10.0,
    rms_norm_eps: float = 1e-20,
    attention_bias: bool = False,
    attention_dropout: float = 0.0,
    initializer_range: float = 0.02,
    use_cache: bool = True,
    tie_word_embeddings: bool = False,
    max_position_embeddings: int = 1048576,
    rope_theta: float = 10000.0,
    rope_scaling: dict[str, typing.Any] | None = None,
    n_routed_experts: int = 384,
    n_shared_experts: int = 1,
    num_experts_per_tok: int = 6,
    scoring_func: str = 'sqrtsoftplus',
    topk_method: str = 'noaux_tc',
    norm_topk_prob: bool = True,
    routed_scaling_factor: float = 1.5,
    sliding_window: int = 128,
    compress_ratios: list[int] | None = None,
    compress_rope_theta: float = 160000.0,
    kv_source_layer_ids: list[int] | None = None,
    index_source_layer_ids: list[int] | None = None,
    index_n_heads: int = 32,
    index_head_dim: int = 128,
    index_topk: int = 512,
    candidate_source_layer_id: int = 20,
    candidate_topk_blocks: int = 2048,
    candidate_block_size: int = 8,
    hc_mult: int = 4,
    hc_sinkhorn_iters: int = 20,
    hc_eps: float = 1e-06,
    engram_layer_ids: list[int] | None = None,
    engram_num_embeddings: list[int] | None = None,
    engram_max_ngram_size: int = 4,
    engram_vocab_size: int = 16000000,
    engram_n_heads: int = 8,
    engram_head_dim: int = 256,
    engram_pad_token_id: int = 2,
    engram_compressed_vocab_size: int = 99092,
    num_nextn_predict_layers: int = 3,
    dspark_block_size: int = 5,
    dspark_noise_token_id: int = 128799,
    dspark_target_layer_ids: list[int] | None = None,
    dspark_markov_rank: int = 256,
    dspark_n_routed_experts: int = 128,
    dspark_num_experts_per_tok: int = 3,
    dtype: str = 'bfloat16',
    pad_token_id: int | None = 2,
    bos_token_id: int = 0,
    eos_token_id: int = 1,
    kwargs: typing.Any = {}
)
```

**Bases:** `PretrainedConfig`

Declarative configuration of the CSA2, single-pass mHC and Engram backbone.

Defaults match the released Flash text configuration. Reducing
`num_hidden_layers` retains the complete source-layer and compression
schedules: a pretrained prefix must keep its original sharing and hashing
identities. Explicit smaller schedules support independent tiny models.
Unknown checkpoint metadata is retained by the Hugging Face base class.

**`base_config_key`** `= 'text_config'`

---

**`compress_ratios`**

---

**`dspark_target_layer_ids`**

---

**`engram_layer_ids`**

---

**`engram_num_embeddings`**

---

**`index_source_layer_ids`**

---

**`keys_to_ignore_at_inference`** `= ['past_key_values']`

---

**`kv_source_layer_ids`**

---

**`model_type`** `= 'deepseek_v41_text'`

---

**`rope_scaling`**

---

```python
nemo_automodel.components.models.deepseek_v41.config.DeepseekV41TextConfig._validate_dimensions() -> None
```

```python
nemo_automodel.components.models.deepseek_v41.config.DeepseekV41TextConfig._validate_layer_schedule() -> None
```

```python
class nemo_automodel.components.models.deepseek_v41.config.DeepseekV41VisionConfig(
    num_hidden_layers: int = 32,
    hidden_size: int = 1024,
    num_attention_heads: int = 16,
    intermediate_size: int = 2816,
    patch_size: int = 14,
    rope_theta: float = 10000.0,
    downsample_ratio: int = 3,
    max_image_tokens: int = 1024,
    min_pixels: int = 295936,
    max_wh_ratio: float | None = None,
    kwargs: typing.Any = {}
)
```

**Bases:** `PretrainedConfig`

Configuration of the released 2D-RoPE vision encoder and image sizing.

**`base_config_key`** `= 'vision_config'`

---

**`model_type`** `= 'deepseek_v41_vision'`

---

```python
class nemo_automodel.components.models.deepseek_v41.config._DSparkDraftOptions()
```

Protocol

Declarative DSpark settings supplied by the generic training recipe.

**`block_size`** `int`

---

**`confidence_head_alpha`** `float`

---

**`confidence_head_stop_gradient`** `bool`

---

**`confidence_head_with_markov`** `bool`

---

**`markov_head_type`** `str`

---

**`markov_rank`** `int`

---

**`mask_token_id`** `int`

---

**`num_anchors`** `int`

---

**`num_draft_layers`** `int`

---

**`target_layer_ids`** `list[int]`

---