> This page is for version 0.5.0 · 26.06.
> For other versions, use one of these documentation indexes:
> - Nightly (default): https://docs.nvidia.com/nemo/automodel/nightly/llms.txt
> - Latest: https://docs.nvidia.com/nemo/automodel/latest/llms.txt
> - 0.5.0 · 26.06: https://docs.nvidia.com/nemo/automodel/v0.5/llms.txt
> - 0.4.0 · 26.04: https://docs.nvidia.com/nemo/automodel/v0.4/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.models.deepseek_v4.model

DeepSeek V4 Model.

Key architectural points (from official inference/model.py):

HC (Hyper-Connections):
Every transformer block maintains hc\_mult=4 copies of the hidden state.
The embedding output is expanded: \[B,S,dim] -> \[B,S,hc\_mult,dim].
hc\_pre  reduces \[B,S,hc\_mult,dim] -> \[B,S,dim] before attn/ffn.
hc\_post expands \[B,S,dim] -> \[B,S,hc\_mult,dim] after attn/ffn.
Full HC requires the hc\_split\_sinkhorn CUDA kernel.
Current fallback: mean-pooling for hc\_pre, broadcast add for hc\_post.

HC parameters (ALL layers, stored in float32):
hc\_attn\_fn    : \[mix\_hc, hc\_mult\*dim]  where mix\_hc = (2+hc\_mult)*hc\_mult = 24
hc\_attn\_base  : \[mix\_hc]
hc\_attn\_scale : \[3]
hc\_ffn\_fn     : \[mix\_hc, hc\_mult*dim]
hc\_ffn\_base   : \[mix\_hc]
hc\_ffn\_scale  : \[3]

Gate hash layers (layer\_idx \< num\_hash\_layers):
Instead of score-based routing, the gate uses a fixed token-id -> expert-id
lookup table (tid2eid: \[vocab\_size, n\_activated\_experts]).

All layers use MoE FFN (no dense layers).
Compress-ratio sliding-window attention is not yet implemented.

## Module Contents

### Classes

| Name                                                                                                       | Description                                                                    |
| ---------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ |
| [`DeepseekV4Block`](#nemo_automodel-components-models-deepseek_v4-model-DeepseekV4Block)                   | Single transformer block for DeepSeek V4.                                      |
| [`DeepseekV4CausalLMOutput`](#nemo_automodel-components-models-deepseek_v4-model-DeepseekV4CausalLMOutput) | Output of DeepseekV4ForCausalLM.forward.                                       |
| [`DeepseekV4ForCausalLM`](#nemo_automodel-components-models-deepseek_v4-model-DeepseekV4ForCausalLM)       | -                                                                              |
| [`DeepseekV4HashGate`](#nemo_automodel-components-models-deepseek_v4-model-DeepseekV4HashGate)             | Hash gate for first num\_hash\_layers: routes tokens via a fixed lookup table. |
| [`DeepseekV4Model`](#nemo_automodel-components-models-deepseek_v4-model-DeepseekV4Model)                   | -                                                                              |
| [`DeepseekV4VisionGate`](#nemo_automodel-components-models-deepseek_v4-model-DeepseekV4VisionGate)         | DSV4 gate with separate visual bias and optional text hash routing.            |

### Functions

| Name                                                                                                                           | Description                                                        |
| ------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------ |
| [`_normalize_thd_packing_metadata`](#nemo_automodel-components-models-deepseek_v4-model-_normalize_thd_packing_metadata)       | Accept standard THD offsets at the DSV4 model boundary.            |
| [`_seq_lens_from_cu_seqlens`](#nemo_automodel-components-models-deepseek_v4-model-_seq_lens_from_cu_seqlens)                   | Convert standard THD cumulative offsets to DSV4's per-row lengths. |
| [`apply_deepseek_v4_image_visibility`](#nemo_automodel-components-models-deepseek_v4-model-apply_deepseek_v4_image_visibility) | Make every token inside an image span mutually visible.            |

### Data

[`ModelClass`](#nemo_automodel-components-models-deepseek_v4-model-ModelClass)

### API

```python
class nemo_automodel.components.models.deepseek_v4.model.DeepseekV4Block(
    layer_idx: int,
    config: nemo_automodel.components.models.deepseek_v4.config.DeepseekV4Config,
    moe_config: nemo_automodel.components.moe.config.MoEConfig,
    backend: nemo_automodel.components.models.common.BackendConfig
)
```

**Bases:** `Module`

Single transformer block for DeepSeek V4.

Uses HuggingFace transformers PR 45616's HyperConnection decoder-layer
pattern: two `DeepseekV4HyperConnection` modules own the collapse /
expand mixer weights at the attention and FFN sites respectively.
Checkpoint's flat `hc_attn_*` / `hc_ffn_*` keys are routed into
`attn_hc.*` / `ffn_hc.*` by the state-dict adapter.

**`attn_hc`** `= DeepseekV4HyperConnection(**hc_kwargs)`

---

**`ffn_hc`** `= DeepseekV4HyperConnection(**hc_kwargs)`

---

**`hc_mult`** `= config.hc_mult`

---

**`input_layernorm`**

---

**`is_hash_routing_layer`**

---

**`is_vision_routing`**

---

**`mlp`** `= MoE(moe_config, backend)`

---

**`post_attention_layernorm`**

---

**`self_attn`**

---

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4Block.forward(
    x: torch.Tensor,
    position_embeddings: tuple[torch.Tensor, torch.Tensor],
    position_ids: torch.Tensor | None = None,
    position_embeddings_compress: tuple[torch.Tensor, torch.Tensor] | None = None,
    rotary_compress: torch.nn.Module | None = None,
    attention_mask: torch.Tensor | None = None,
    padding_mask: torch.Tensor | None = None,
    input_ids: torch.Tensor | None = None,
    vision_token_types: torch.Tensor | None = None,
    attn_kwargs: typing.Any = {}
) -> torch.Tensor
```

Transform one HC block.

**Parameters:**

**`x`** `torch.Tensor`

HC streams with layout `[batch, sequence, hc_mult, hidden]`.

---

**`position_embeddings`** `tuple[torch.Tensor, torch.Tensor]`

Main RoPE tensors with layout compatible with
`[batch, sequence, qk_rope_head_dim]`.

---

**`position_ids`** `torch.Tensor | None` — default: None

Token positions with layout `[batch, sequence]`.

---

**`position_embeddings_compress`** `tuple[torch.Tensor, torch.Tensor] | None` — default: None

Optional compressor RoPE tensors.

---

**`rotary_compress`** `nn.Module | None` — default: None

Optional compressor rotary module.

---

**`attention_mask`** `torch.Tensor | None` — default: None

Additive attention mask with layout
`[batch, 1, sequence, sequence]`.

---

**`padding_mask`** `torch.Tensor | None` — default: None

Boolean padding mask with layout `[batch, sequence]`.

---

**`input_ids`** `torch.Tensor | None` — default: None

Token IDs with layout `[batch, sequence]`.

---

**`vision_token_types`** `torch.Tensor | None` — default: None

Visual pseudo-token types with layout
`[batch, sequence]` and `-1` at text positions.

---

**Returns:** `torch.Tensor`

HC streams with layout `[batch, sequence, hc_mult, hidden]`.

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4Block.init_weights(
    buffer_device: torch.device,
    init_std: float = 0.02
) -> None
```

```python
class nemo_automodel.components.models.deepseek_v4.model.DeepseekV4CausalLMOutput(
    mtp_per_depth_h: list[torch.Tensor] | None = None,
    mtp_loss_scaling_factor: float | None = None
)
```

Dataclass

**Bases:** `CausalLMOutputWithPast`

Output of DeepseekV4ForCausalLM.forward.

Subclasses `transformers.modeling_outputs.CausalLMOutputWithPast` so the
standard `logits` / `hidden_states` fields are present (the recipe's
fused cross-entropy path requires `"hidden_states" in out` and reads the
final hidden states off the output) while the DSV4-specific MTP fields are
carried as declared dataclass fields. As required by `ModelOutput`, every
field after the first declares a `None` default.

**`hidden_states`**

Final pre-lm\_head hidden states `[B, S, hidden]`
(or `[T, hidden]` for packed THD), populated only when
`output_hidden_states` is set; otherwise `None`.

---

**`logits`**

`[B, S, vocab_size]` next-token prediction logits.

---

**`mtp_loss_scaling_factor`** `float | None = None`

Coefficient for the MTP auxiliary loss.

---

**`mtp_per_depth_h`** `list[Tensor] | None = None`

Per-depth MTP hidden states (training mode only).
List of length `num_nextn_predict_layers`, each `[B, S, hidden]`.
`None` when MTP is disabled or in eval mode.

---

```python
class nemo_automodel.components.models.deepseek_v4.model.DeepseekV4ForCausalLM(
    config: nemo_automodel.components.models.deepseek_v4.config.DeepseekV4Config,
    moe_config: nemo_automodel.components.moe.config.MoEConfig | None = None,
    backend: nemo_automodel.components.models.common.BackendConfig | None = None,
    kwargs = {}
)
```

**Bases:** [HFCheckpointingMixin](/nemo-automodel/nemo_automodel/components/models/common/hf_checkpointing_mixin#nemo_automodel-components-models-common-hf_checkpointing_mixin-HFCheckpointingMixin), `Module`, [MoEFSDPSyncMixin](/nemo-automodel/nemo_automodel/components/moe/fsdp_mixin#nemo_automodel-components-moe-fsdp_mixin-MoEFSDPSyncMixin)

**`_keep_in_fp32_modules_strict`**

---

**`backend`** `= backend or BackendConfig()`

---

**`lm_head`**

---

**`model`**

---

**`mtp`**

---

**`mtp_config`**

---

**`state_dict_adapter`**

---

**`tie_word_embeddings_support`** `TieSupport = TieSupport.UNTIED_ONLY`

---

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4ForCausalLM._build_mtp_embed_inputs_for_pp(
    input_ids: torch.Tensor
) -> tuple[torch.Tensor, ...]
```

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4ForCausalLM._is_pipeline_parallel_stage() -> bool
```

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4ForCausalLM.customize_pipeline_stage_modules(
    module_names_per_stage: list[list[str]],
    layers_prefix: str,
    text_model: torch.nn.Module | None = None
) -> list[list[str]]
```

Keep DSV4 non-layer PP dependencies with the stages that need them.

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4ForCausalLM.forward(
    input_ids: torch.Tensor | None = None,
    mtp_embed_inputs: torch.Tensor = (),
    position_ids: torch.Tensor | None = None,
    attention_mask: torch.Tensor | None = None,
    padding_mask: torch.Tensor | None = None,
    pixel_values: torch.Tensor | None = None,
    image_grid_hws: torch.Tensor | None = None,
    n_images_per_sample: torch.Tensor | None = None,
    vision_token_types: torch.Tensor | None = None,
    logits_to_keep: typing.Union[int, torch.Tensor] = 0,
    output_hidden_states: bool | None = None,
    attn_kwargs: typing.Any = {}
) -> 'DeepseekV4CausalLMOutput' | tuple[torch.Tensor, ...] | torch.Tensor
```

Run causal language modeling with optional DSV4 visual inputs.

**Parameters:**

**`input_ids`** `torch.Tensor | None` — default: None

Token IDs with layout `[batch, sequence]`.

---

**`mtp_embed_inputs`** `torch.Tensor` — default: ()

Optional PP-propagated MTP embeddings, each with
layout `[batch, sequence, hidden]`.

---

**`position_ids`** `torch.Tensor | None` — default: None

Position IDs with layout `[batch, sequence]`.

---

**`attention_mask`** `torch.Tensor | None` — default: None

Valid-token mask with layout `[batch, sequence]`.

---

**`padding_mask`** `torch.Tensor | None` — default: None

Padding mask with layout `[batch, sequence]`.

---

**`pixel_values`** `torch.Tensor | None` — default: None

Concatenated patches with layout
`[all_patches, 3, patch_size, patch_size]`.

---

**`image_grid_hws`** `torch.Tensor | None` — default: None

ViT grids with layout `[all_images, 2]`.

---

**`n_images_per_sample`** `torch.Tensor | None` — default: None

Image counts with layout `[batch]`.

---

**`vision_token_types`** `torch.Tensor | None` — default: None

Visual types with layout `[batch, sequence]`
and `-1` for text.

---

**`logits_to_keep`** `Union[int, torch.Tensor]` — default: 0

Number or positions of logits to retain.

---

**`output_hidden_states`** `bool | None` — default: None

Whether to expose final states.

---

**Returns:** `'DeepseekV4CausalLMOutput' | tuple[torch.Tensor, ...] | torch.Tensor`

`DeepseekV4CausalLMOutput` outside PP, or the PP stage tensor

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4ForCausalLM.from_config(
    config: nemo_automodel.components.models.deepseek_v4.config.DeepseekV4Config,
    moe_config: nemo_automodel.components.moe.config.MoEConfig | None = None,
    backend: nemo_automodel.components.models.common.BackendConfig | None = None,
    kwargs = {}
)
```

classmethod

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4ForCausalLM.from_pretrained(
    pretrained_model_name_or_path: str,
    model_args = (),
    kwargs = {}
)
```

classmethod

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4ForCausalLM.get_input_embeddings()
```

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4ForCausalLM.get_output_embeddings()
```

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4ForCausalLM.get_pipeline_stage_metas(
    is_first: bool,
    microbatch_size: int,
    seq_len: int,
    dtype: torch.dtype
) -> tuple[tuple[torch.Tensor, ...], tuple[torch.Tensor, ...]]
```

Return PP input/output meta tensors for DSV4's HC and MTP contract.

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4ForCausalLM.initialize_weights(
    buffer_device: torch.device | None = None,
    dtype: torch.dtype = torch.bfloat16
) -> None
```

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4ForCausalLM.prepare_model_inputs_for_cp(
    batch: dict[str, typing.Any],
    num_chunks: int = 1
) -> dict[str, typing.Any]
```

Model-owned context-parallel batch prep (Miles-style contiguous shard).

Returns a `ContextParallelSharder` (under the `"cp_sharder"` batch key) so
the CP dispatch delegates CP sharding back to this
model, with the config-derived per-rank shard multiple bound. DSV4
embeds internally, so (unlike VLM models) this does not pre-embed --
it leaves `input_ids` for the sharding callable.

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4ForCausalLM.set_input_embeddings(
    value
)
```

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4ForCausalLM.set_output_embeddings(
    new_embeddings
)
```

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4ForCausalLM.update_moe_gate_bias() -> None
```

```python
class nemo_automodel.components.models.deepseek_v4.model.DeepseekV4HashGate(
    config: nemo_automodel.components.models.deepseek_v4.config.DeepseekV4Config,
    moe_config: nemo_automodel.components.moe.config.MoEConfig
)
```

**Bases:** `Module`

Hash gate for first num\_hash\_layers: routes tokens via a fixed lookup table.

Instead of computing routing scores, the gate uses tid2eid\[token\_id] to
pre-assign expert indices.  The routing weight is still computed from the
gate weight but the *selection* is deterministic per token id.

tid2eid shape: \[vocab\_size, n\_activated\_experts]  (int64 runtime, non-trainable)

Signature matches `components.moe.layers.Gate` — `forward(x, token_mask,
cp_mesh)` returning `(weights, indices, aux_loss)` — so the generic MoE
module can call it interchangeably.  The per-forward `input_ids` needed
for the tid2eid lookup is stashed on the module by the enclosing Block via
`set_input_ids` immediately before the MoE call.

**`_pending_input_ids`** `Tensor | None = None`

---

**`n_experts`** `= moe_config.n_routed_experts`

---

**`norm_topk_prob`** `= moe_config.norm_topk_prob`

---

**`route_scale`** `= moe_config.route_scale`

---

**`score_func`** `= moe_config.score_func`

---

**`topk`** `= moe_config.n_activated_experts`

---

**`weight`**

---

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4HashGate.forward(
    x: torch.Tensor,
    token_mask: torch.Tensor | None = None,
    cp_mesh: 'DeviceMesh | None' = None
) -> tuple[torch.Tensor, torch.Tensor, None]
```

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4HashGate.init_weights(
    init_std: float = 0.02
) -> None
```

Initialize the trainable gate and a valid deterministic hash table.

**Parameters:**

**`init_std`** `float` — default: 0.02

Standard deviation for the routing weight initialization.

---

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4HashGate.set_input_ids(
    input_ids: torch.Tensor | None
) -> None
```

Stash the current batch's input\_ids for the next `forward` call.

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4HashGate.update_bias() -> None
```

No-op for compat with callers that walk MoE gates and call update\_bias.

```python
class nemo_automodel.components.models.deepseek_v4.model.DeepseekV4Model(
    config: nemo_automodel.components.models.deepseek_v4.config.DeepseekV4Config,
    backend: nemo_automodel.components.models.common.BackendConfig,
    moe_config: nemo_automodel.components.moe.config.MoEConfig | None = None,
    moe_overrides: dict | None = None
)
```

**Bases:** `Module`

**`aligner`** `= DeepseekV4VisionAligner(config)`

---

**`embed_tokens`**

---

**`hc_head`**

---

**`image_end`**

---

**`image_newline`**

---

**`image_pad`**

---

**`image_start`**

---

**`layers`** `= nn.ModuleDict()`

---

**`max_seq_len`** `= config.max_position_embeddings`

---

**`moe_config`** `= moe_config or MoEConfig(**moe_defaults)`

---

**`norm`**

---

**`rotary_emb`**

---

**`rotary_emb_compress`**

---

**`vision`** `= DeepseekV4VisionTransformer(config)`

---

**`vision_enabled`**

---

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4Model.encode_image(
    patches: torch.Tensor,
    n_vit_h: int,
    n_vit_w: int
) -> torch.Tensor
```

Encode one image from ViT patches into LLM-width grid features.

**Parameters:**

**`patches`** `torch.Tensor`

Image patches with layout
`[n_vit_h * n_vit_w, 3, patch_size, patch_size]`.

---

**`n_vit_h`** `int`

Number of patch rows.

---

**`n_vit_w`** `int`

Number of patch columns.

---

**Returns:** `torch.Tensor`

Aligned features with layout

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4Model.forward(
    input_ids: torch.Tensor | None = None,
    inputs_embeds: torch.Tensor | None = None,
    position_ids: torch.Tensor | None = None,
    attention_mask: torch.Tensor | None = None,
    padding_mask: torch.Tensor | None = None,
    pixel_values: torch.Tensor | None = None,
    image_grid_hws: torch.Tensor | None = None,
    n_images_per_sample: torch.Tensor | None = None,
    vision_token_types: torch.Tensor | None = None,
    return_hc_hidden: bool = False,
    attn_kwargs: typing.Any = {}
) -> torch.Tensor | tuple[torch.Tensor, torch.Tensor]
```

Run the DSV4 text backbone with an optional image embedding bridge.

**Parameters:**

**`input_ids`** `torch.Tensor | None` — default: None

Token IDs with layout `[batch, sequence]` on the first
PP stage, or HC activations on later stages.

---

**`inputs_embeds`** `torch.Tensor | None` — default: None

Optional embeddings with layout
`[batch, sequence, hidden]`.

---

**`position_ids`** `torch.Tensor | None` — default: None

Positions with layout `[batch, sequence]`.

---

**`attention_mask`** `torch.Tensor | None` — default: None

Valid-token mask with layout `[batch, sequence]`.

---

**`padding_mask`** `torch.Tensor | None` — default: None

Padding mask with layout `[batch, sequence]`.

---

**`pixel_values`** `torch.Tensor | None` — default: None

Concatenated image patches with layout
`[all_patches, 3, patch_size, patch_size]`.

---

**`image_grid_hws`** `torch.Tensor | None` — default: None

Patch grids with layout `[all_images, 2]`.

---

**`n_images_per_sample`** `torch.Tensor | None` — default: None

Image counts with layout `[batch]`.

---

**`vision_token_types`** `torch.Tensor | None` — default: None

Pseudo-token types with layout
`[batch, sequence]` and `-1` at text positions.

---

**`return_hc_hidden`** `bool` — default: False

Whether to also return the uncollapsed HC stream.

---

**Returns:** `torch.Tensor | tuple[torch.Tensor, torch.Tensor]`

Hidden states with layout `[batch, sequence, hidden]` and,

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4Model.init_weights(
    buffer_device: torch.device | None = None
) -> None
```

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4Model.merge_image_embeddings(
    inputs_embeds: torch.Tensor,
    pixel_values: torch.Tensor,
    image_grid_hws: torch.Tensor,
    vision_token_types: torch.Tensor,
    n_images_per_sample: torch.Tensor | None
) -> torch.Tensor
```

Replace pseudo-token embeddings with encoded images and sentinels.

**Parameters:**

**`inputs_embeds`** `torch.Tensor`

Text embeddings with layout `[batch, sequence, hidden]`.

---

**`pixel_values`** `torch.Tensor`

Concatenated patches with layout
`[all_patches, 3, patch_size, patch_size]`.

---

**`image_grid_hws`** `torch.Tensor`

ViT grid sizes with layout `[all_images, 2]`.

---

**`vision_token_types`** `torch.Tensor`

Pseudo-token types with layout
`[batch, sequence]` and `-1` for text.

---

**`n_images_per_sample`** `torch.Tensor | None`

Optional counts with layout `[batch]`.

---

**Returns:** `torch.Tensor`

Embeddings with the same layout as `inputs_embeds`.

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4Model.update_moe_gate_bias() -> None
```

Update routing biases only when enabled by the model's MoE configuration.

```python
class nemo_automodel.components.models.deepseek_v4.model.DeepseekV4VisionGate(
    config: nemo_automodel.components.models.deepseek_v4.config.DeepseekV4Config,
    moe_config: nemo_automodel.components.moe.config.MoEConfig,
    gate_precision: torch.dtype | None,
    hash_routing: bool
)
```

**Bases:** [Gate](/nemo-automodel/nemo_automodel/components/moe/layers#nemo_automodel-components-moe-layers-Gate)

DSV4 gate with separate visual bias and optional text hash routing.

The released vision checkpoint routes visual pseudo tokens by score in all
layers. In the first hash layers only text tokens use `tid2eid`; visual
tokens use `scores + bias_vl`. Later layers select text experts with the
normal correction bias and visual experts with `bias_vl`.

**`_pending_input_ids`** `Tensor | None = None`

---

**`_pending_vision_token_types`** `Tensor | None = None`

---

**`vocab_size`** `= int(config.vocab_size)`

---

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4VisionGate._local_tensor(
    tensor: torch.Tensor
) -> torch.Tensor
```

staticmethod

Return a local `[experts]` tensor from a tensor or DTensor.

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4VisionGate._route_scores(
    scores: torch.Tensor
) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]
```

Select experts from router logits of layout `[tokens, experts]`.

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4VisionGate.init_dsv4_weights() -> None
```

Initialize visual bias and a deterministic checkpoint-free hash map.

```python
nemo_automodel.components.models.deepseek_v4.model.DeepseekV4VisionGate.set_routing_context(
    input_ids: torch.Tensor | None,
    vision_token_types: torch.Tensor | None
) -> None
```

Set token metadata consumed by the next gate call.

**Parameters:**

**`input_ids`** `torch.Tensor | None`

Token IDs with layout `[batch, sequence]`.

---

**`vision_token_types`** `torch.Tensor | None`

Visual types with layout `[batch, sequence]`
and `-1` for text tokens.

---

```python
nemo_automodel.components.models.deepseek_v4.model._normalize_thd_packing_metadata(
    attn_kwargs: dict[str, typing.Any]
) -> None
```

Accept standard THD offsets at the DSV4 model boundary.

DSV4 internally uses `seq_lens` to build document-aware masks. Packed
callers commonly provide the equivalent `cu_seqlens` representation, so
normalize it here when context parallelism has not already produced native
padded-BSHD lengths.

```python
nemo_automodel.components.models.deepseek_v4.model._seq_lens_from_cu_seqlens(
    cu_seqlens: torch.Tensor,
    name: str
) -> torch.Tensor
```

Convert standard THD cumulative offsets to DSV4's per-row lengths.

```python
nemo_automodel.components.models.deepseek_v4.model.apply_deepseek_v4_image_visibility(
    attention_mask: torch.Tensor,
    vision_token_types: torch.Tensor
) -> torch.Tensor
```

Make every token inside an image span mutually visible.

**Parameters:**

**`attention_mask`** `torch.Tensor`

Additive causal/sliding mask with layout
`[batch, 1, sequence, sequence]`.

---

**`vision_token_types`** `torch.Tensor`

Pseudo-token types with layout `[batch, sequence]`
and `-1` at text positions.

---

**Returns:** `torch.Tensor`

Additive mask with the same layout as `attention_mask`. Text

```python
nemo_automodel.components.models.deepseek_v4.model.ModelClass = DeepseekV4ForCausalLM
```