> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.models.kimi_k25_vl.model

KimiK25VL model with backend-aware DeepseekV3 language model.

This is a self-contained implementation that includes all necessary components:

* Configuration classes
* Vision tower (MoonViT3d with temporal dimension)
* Multi-modal projector (PatchMergerMLP)
* Language model backend (DeepseekV3)

## Module Contents

### Classes

| Name                                                                                                                             | Description                                                                          |
| -------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
| [`DeepSeekV3RotaryEmbeddingAdapter`](#nemo_automodel-components-models-kimi_k25_vl-model-DeepSeekV3RotaryEmbeddingAdapter)       | Callable adapter that wraps DeepseekV3's freqs\_cis-based RoPE.                      |
| [`KimiK25VLConfig`](#nemo_automodel-components-models-kimi_k25_vl-model-KimiK25VLConfig)                                         | Configuration for KimiK25VL model.                                                   |
| [`KimiK25VLForConditionalGeneration`](#nemo_automodel-components-models-kimi_k25_vl-model-KimiK25VLForConditionalGeneration)     | KimiK25VL model with backend-aware DeepseekV3 language model.                        |
| [`KimiK25VLLanguageModelBackend`](#nemo_automodel-components-models-kimi_k25_vl-model-KimiK25VLLanguageModelBackend)             | Backend-aware language model wrapper using DeepseekV3 architecture.                  |
| [`KimiK25VLModel`](#nemo_automodel-components-models-kimi_k25_vl-model-KimiK25VLModel)                                           | KimiK25VL multimodal backbone with a DeepseekV3 text decoder.                        |
| [`KimiK25VLMultiModalProjector`](#nemo_automodel-components-models-kimi_k25_vl-model-KimiK25VLMultiModalProjector)               | Projects vision features to language model dimension using patch merger MLP.         |
| [`Learnable2DInterpPosEmbDividedFixed`](#nemo_automodel-components-models-kimi_k25_vl-model-Learnable2DInterpPosEmbDividedFixed) | Learnable 2D interpolatable position embedding with fixed temporal sincos embedding. |
| [`MoonViT3dConfig`](#nemo_automodel-components-models-kimi_k25_vl-model-MoonViT3dConfig)                                         | Configuration for MoonViT3d vision encoder with temporal support.                    |
| [`MoonViT3dEncoder`](#nemo_automodel-components-models-kimi_k25_vl-model-MoonViT3dEncoder)                                       | MoonViT3d encoder with temporal support.                                             |
| [`MoonViT3dEncoderLayer`](#nemo_automodel-components-models-kimi_k25_vl-model-MoonViT3dEncoderLayer)                             | Single encoder layer for MoonViT3d.                                                  |
| [`MoonViT3dMLP`](#nemo_automodel-components-models-kimi_k25_vl-model-MoonViT3dMLP)                                               | MLP for MoonViT3d.                                                                   |
| [`MoonViT3dPretrainedModel`](#nemo_automodel-components-models-kimi_k25_vl-model-MoonViT3dPretrainedModel)                       | MoonViT3d vision encoder with temporal support.                                      |
| [`MoonVision3dPatchEmbed`](#nemo_automodel-components-models-kimi_k25_vl-model-MoonVision3dPatchEmbed)                           | Patch embedding for MoonViT3d.                                                       |
| [`Rope2DPosEmbRepeated`](#nemo_automodel-components-models-kimi_k25_vl-model-Rope2DPosEmbRepeated)                               | 2D rotary position embedding repeated for temporal dimension.                        |

### Functions

| Name                                                                                                                                     | Description                                                         |
| ---------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
| [`_apply_rope_vision`](#nemo_automodel-components-models-kimi_k25_vl-model-_apply_rope_vision)                                           | Apply rotary position embedding for vision.                         |
| [`_register_kimi_k25_vl_with_transformers`](#nemo_automodel-components-models-kimi_k25_vl-model-_register_kimi_k25_vl_with_transformers) | Register KimiK25VLConfig and model with transformers Auto classes.  |
| [`compute_expanded_seq_length`](#nemo_automodel-components-models-kimi_k25_vl-model-compute_expanded_seq_length)                         | Compute the expanded sequence length after image token insertion.   |
| [`get_1d_sincos_pos_embed`](#nemo_automodel-components-models-kimi_k25_vl-model-get_1d_sincos_pos_embed)                                 | Generate 1D sinusoidal positional embedding for temporal dimension. |
| [`get_1d_sincos_pos_embed_from_grid`](#nemo_automodel-components-models-kimi_k25_vl-model-get_1d_sincos_pos_embed_from_grid)             | Generate 1D sinusoidal positional embedding from grid positions.    |
| [`tpool_patch_merger`](#nemo_automodel-components-models-kimi_k25_vl-model-tpool_patch_merger)                                           | Merge patches with temporal pooling.                                |
| [`vision_attention_flash`](#nemo_automodel-components-models-kimi_k25_vl-model-vision_attention_flash)                                   | Flash attention for vision.                                         |
| [`vision_attention_sdpa`](#nemo_automodel-components-models-kimi_k25_vl-model-vision_attention_sdpa)                                     | SDPA attention for vision.                                          |

### Data

[`FLASH_ATTN_AVAILABLE`](#nemo_automodel-components-models-kimi_k25_vl-model-FLASH_ATTN_AVAILABLE)

[`LOGGER`](#nemo_automodel-components-models-kimi_k25_vl-model-LOGGER)

[`ModelClass`](#nemo_automodel-components-models-kimi_k25_vl-model-ModelClass)

### API

```python
class nemo_automodel.components.models.kimi_k25_vl.model.DeepSeekV3RotaryEmbeddingAdapter(
    parent_module: torch.nn.Module,
    rope_fusion: bool = False
)
```

Callable adapter that wraps DeepseekV3's freqs\_cis-based RoPE.

```python
nemo_automodel.components.models.kimi_k25_vl.model.DeepSeekV3RotaryEmbeddingAdapter.__call__(
    hidden_states: torch.Tensor,
    position_ids: torch.Tensor
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLConfig(
    vision_config: typing.Union[typing.Dict, nemo_automodel.components.models.kimi_k25_vl.model.MoonViT3dConfig] | None = None,
    text_config: typing.Union[typing.Dict, transformers.models.deepseek_v3.configuration_deepseek_v3.DeepseekV3Config] | None = None,
    ignore_index: int = -100,
    media_placeholder_token_id: int = 163605,
    pad_token_id: int = 0,
    tie_word_embeddings: bool = False,
    mm_projector_type: str = 'patchmerger',
    mm_hidden_size: int | None = None,
    projector_hidden_act: str = 'gelu',
    projector_ln_eps: float = 1e-05,
    architectures: typing.List[str] | None = None,
    kwargs = {}
)
```

**Bases:** `PretrainedConfig`

Configuration for KimiK25VL model.

Supports both 'kimi\_k25\_vl' and 'kimi\_k25' model types for compatibility
with original checkpoints.

**`mm_hidden_size`**

---

**`model_type`** `= 'kimi_k25'`

---

```python
nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLConfig.to_dict() -> typing.Dict[str, typing.Any]
```

```python
class nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLForConditionalGeneration(
    config,
    moe_config: nemo_automodel.components.moe.config.MoEConfig | None = None,
    backend: nemo_automodel.components.models.common.BackendConfig | None = None,
    kwargs = {}
)
```

**Bases:** [HFCheckpointingMixin](/nemo-automodel/nemo_automodel/components/models/common/hf_checkpointing_mixin#nemo_automodel-components-models-common-hf_checkpointing_mixin-HFCheckpointingMixin), `Module`, [MoEFSDPSyncMixin](/nemo-automodel/nemo_automodel/components/moe/fsdp_mixin#nemo_automodel-components-moe-fsdp_mixin-MoEFSDPSyncMixin)

KimiK25VL model with backend-aware DeepseekV3 language model.

**`_keep_in_fp32_modules`** `= ['freqs_cis', 'rotary_emb']`

---

**`_no_split_modules`** `= ['MoonViT3dEncoderLayer']`

---

**`_pp_keep_self_forward`** `bool = True`

---

**`backend`** `= backend or BackendConfig()`

---

**`base_model_prefix`** `= 'model'`

---

**`main_input_name`** `= 'pixel_values'`

---

**`media_placeholder_token_id`** `= config.media_placeholder_token_id`

---

**`model`**

---

**`moe_config`** `= self.model.moe_config`

---

**`pad_token_id`**

---

**`state_dict_adapter`**

---

**`tie_word_embeddings_support`** `TieSupport = TieSupport.UNTIED_ONLY`

---

**`vocab_size`** `= config.text_config.vocab_size`

---

```python
nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLForConditionalGeneration.forward(
    input_ids = None,
    attention_mask = None,
    position_ids = None,
    past_key_values = None,
    inputs_embeds = None,
    labels = None,
    use_cache = None,
    output_attentions = None,
    output_hidden_states: bool | None = None,
    return_dict = None,
    pixel_values = None,
    grid_thws = None,
    padding_mask = None,
    target_seq_length = None,
    logits_to_keep: typing.Union[int, torch.Tensor] = 0,
    kwargs = {}
)
```

```python
nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLForConditionalGeneration.from_config(
    config,
    moe_config: nemo_automodel.components.moe.config.MoEConfig | None = None,
    backend: nemo_automodel.components.models.common.BackendConfig | None = None,
    kwargs = {}
)
```

classmethod

```python
nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLForConditionalGeneration.from_pretrained(
    pretrained_model_name_or_path: str,
    model_args = (),
    kwargs = {}
)
```

classmethod

Load model from pretrained path.

Creates the model structure. Weights are loaded by DCP which calls the
state\_dict\_adapter.to\_hf() to get checkpoint-format keys (including
*\_packed/*\_scale/\*\_shape for INT4), then from\_hf() to dequantize.

```python
nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLForConditionalGeneration.get_input_embeddings()
```

```python
nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLForConditionalGeneration.get_output_embeddings()
```

```python
nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLForConditionalGeneration.initialize_weights(
    buffer_device = None,
    dtype = torch.bfloat16
)
```

```python
nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLForConditionalGeneration.set_input_embeddings(
    value
)
```

```python
nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLForConditionalGeneration.set_output_embeddings(
    new_embeddings
)
```

```python
class nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLLanguageModelBackend(
    config,
    backend: nemo_automodel.components.models.common.BackendConfig,
    moe_config: nemo_automodel.components.moe.config.MoEConfig | None = None
)
```

**Bases:** `Module`

Backend-aware language model wrapper using DeepseekV3 architecture.

**`model`**

---

**`moe_config`** `= self.model.moe_config`

---

**`rotary_emb`**

---

```python
nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLLanguageModelBackend.forward(
    input_ids = None,
    inputs_embeds = None,
    attention_mask = None,
    position_ids = None,
    padding_mask = None,
    kwargs = {}
)
```

```python
nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLLanguageModelBackend.get_input_embeddings()
```

```python
nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLLanguageModelBackend.init_weights(
    buffer_device = None
)
```

```python
nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLLanguageModelBackend.set_input_embeddings(
    value
)
```

```python
class nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLModel(
    config,
    moe_config: nemo_automodel.components.moe.config.MoEConfig | None = None,
    backend: nemo_automodel.components.models.common.BackendConfig | None = None
)
```

**Bases:** `Module`

KimiK25VL multimodal backbone with a DeepseekV3 text decoder.

**`backend`** `= backend or BackendConfig()`

---

**`language_model`**

---

**`media_placeholder_token_id`** `= config.media_placeholder_token_id`

---

**`moe_config`** `= self.language_model.moe_config`

---

**`multi_modal_projector`** `= KimiK25VLMultiModalProjector(config)`

---

**`vision_tower`** `= MoonViT3dPretrainedModel(config.vision_config)`

---

```python
nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLModel._compute_num_image_tokens_from_grid(
    grid_thws: torch.Tensor
) -> typing.List[int]
```

Pre-compute number of image tokens from grid\_thws without running vision tower.

For 1 image per sample: num\_tokens = (h // merge\_h) \* (w // merge\_w)
With default merge\_kernel\_size=(2,2): num\_tokens = (h // 2) \* (w // 2)

**Parameters:**

**`grid_thws`** `torch.Tensor`

Tensor of shape (batch\_size, 3) with \[t, h, w] per sample

---

**Returns:** `List[int]`

List of expected image token counts per sample

```python
nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLModel._extract_image_features(
    pixel_values,
    grid_thws
)
```

Extract and project image features.

```python
nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLModel._merge_input_ids_with_image_features(
    image_features: typing.List[torch.Tensor],
    inputs_embeds: torch.Tensor,
    input_ids: torch.Tensor,
    attention_mask: torch.Tensor,
    labels: torch.Tensor | None = None,
    target_seq_length: int | None = None
)
```

Merge image features into input embeddings.

Supports two modes:

1. Pre-expanded (PP mode): input\_ids already has N placeholder tokens per image,
   where N = number of image features. Does simple 1:1 replacement.
2. Dynamic expansion: input\_ids has 1 placeholder per image, expands to N tokens.

**Parameters:**

**`image_features`** `List[torch.Tensor]`

List of image feature tensors, one per image

---

**`inputs_embeds`** `torch.Tensor`

Text embeddings (batch\_size, seq\_len, embed\_dim)

---

**`input_ids`** `torch.Tensor`

Token IDs (batch\_size, seq\_len)

---

**`attention_mask`** `torch.Tensor`

Attention mask (batch\_size, seq\_len)

---

**`labels`** `torch.Tensor | None` — default: None

Optional labels for training

---

**`target_seq_length`** `int | None` — default: None

Optional fixed output length for pipeline parallelism.

---

```python
nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLModel.forward(
    input_ids = None,
    attention_mask = None,
    position_ids = None,
    inputs_embeds = None,
    pixel_values = None,
    grid_thws = None,
    labels = None,
    padding_mask = None,
    target_seq_length = None,
    kwargs = {}
)
```

Forward pass with optional fixed sequence length for pipeline parallelism.

**Parameters:**

**`target_seq_length`** — default: None

If provided, the output after image token expansion will be
padded to this fixed length. Required for PP with varying image sizes.
Can be pre-computed as: max\_text\_len - 1 + max\_image\_tokens
where max\_image\_tokens = (h // 2) \* (w // 2) for each image.

---

```python
class nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLMultiModalProjector(
    config
)
```

**Bases:** `Module`

Projects vision features to language model dimension using patch merger MLP.

**`act`** `= GELUActivation()`

---

**`hidden_size`**

---

**`linear_1`**

---

**`linear_2`**

---

**`pre_norm`**

---

```python
nemo_automodel.components.models.kimi_k25_vl.model.KimiK25VLMultiModalProjector.forward(
    image_features: typing.List[torch.Tensor]
) -> typing.List[torch.Tensor]
```

```python
class nemo_automodel.components.models.kimi_k25_vl.model.Learnable2DInterpPosEmbDividedFixed(
    height: int,
    width: int,
    num_frames: int,
    dim: int,
    interpolation_mode: str = 'bicubic'
)
```

**Bases:** `Module`

Learnable 2D interpolatable position embedding with fixed temporal sincos embedding.

**`weight`** `= nn.Parameter(torch.empty(height, width, dim))`

---

```python
nemo_automodel.components.models.kimi_k25_vl.model.Learnable2DInterpPosEmbDividedFixed.forward(
    x: torch.Tensor,
    grid_thws: torch.Tensor
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.kimi_k25_vl.model.MoonViT3dConfig(
    patch_size: int = 14,
    init_pos_emb_height: int = 64,
    init_pos_emb_width: int = 64,
    init_pos_emb_time: int = 4,
    pos_emb_type: str = 'divided_fixed',
    num_attention_heads: int = 16,
    num_hidden_layers: int = 27,
    hidden_size: int = 1152,
    intermediate_size: int = 4304,
    merge_kernel_size: typing.Tuple[int, int] = (2, 2),
    video_attn_type: str = 'spatial_temporal',
    merge_type: str = 'sd2_tpool',
    kwargs = {}
)
```

**Bases:** `PretrainedConfig`

Configuration for MoonViT3d vision encoder with temporal support.

**`merge_kernel_size`**

---

**`model_type`** `= 'moonvit3d'`

---

```python
class nemo_automodel.components.models.kimi_k25_vl.model.MoonViT3dEncoder(
    hidden_dim: int,
    num_layers: int,
    block_cfg: dict
)
```

**Bases:** `Module`

MoonViT3d encoder with temporal support.

**`blocks`**

---

**`final_layernorm`** `= nn.LayerNorm(hidden_dim)`

---

**`rope_2d`**

---

```python
nemo_automodel.components.models.kimi_k25_vl.model.MoonViT3dEncoder.forward(
    hidden_states: torch.Tensor,
    grid_thws: torch.Tensor
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.kimi_k25_vl.model.MoonViT3dEncoderLayer(
    num_heads: int,
    hidden_dim: int,
    mlp_dim: int,
    activation = F.gelu,
    attn_bias: bool = False,
    attn_implementation: str = 'flash_attention_2'
)
```

**Bases:** `Module`

Single encoder layer for MoonViT3d.

**`head_dim`** `= hidden_dim // num_heads`

---

**`mlp`**

---

**`norm0`** `= nn.LayerNorm(hidden_dim)`

---

**`norm1`** `= nn.LayerNorm(hidden_dim)`

---

**`wo`** `= nn.Linear(hidden_dim, hidden_dim, bias=attn_bias)`

---

**`wqkv`**

---

```python
nemo_automodel.components.models.kimi_k25_vl.model.MoonViT3dEncoderLayer.forward(
    hidden_states: torch.Tensor,
    cu_seqlens: torch.Tensor,
    max_seqlen: int,
    rope_freqs_cis: torch.Tensor
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.kimi_k25_vl.model.MoonViT3dMLP(
    dims: typing.List[int],
    activation,
    bias: bool = True
)
```

**Bases:** `Module`

MLP for MoonViT3d.

**`fc0`** `= nn.Linear(dims[0], dims[1], bias=bias)`

---

**`fc1`** `= nn.Linear(dims[1], dims[2], bias=bias)`

---

```python
nemo_automodel.components.models.kimi_k25_vl.model.MoonViT3dMLP.forward(
    x: torch.Tensor
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.kimi_k25_vl.model.MoonViT3dPretrainedModel(
    config
)
```

**Bases:** `Module`

MoonViT3d vision encoder with temporal support.

**`encoder`**

---

**`merge_kernel_size`** `= config.merge_kernel_size`

---

**`merge_type`** `= config.merge_type`

---

**`patch_embed`**

---

```python
nemo_automodel.components.models.kimi_k25_vl.model.MoonViT3dPretrainedModel.forward(
    pixel_values: torch.Tensor,
    grid_thws: torch.Tensor
) -> typing.List[torch.Tensor]
```

```python
class nemo_automodel.components.models.kimi_k25_vl.model.MoonVision3dPatchEmbed(
    out_dim: int,
    in_dim: int = 3,
    patch_size: int = 14,
    pos_emb_height: int = 64,
    pos_emb_width: int = 64,
    pos_emb_time: int = 4
)
```

**Bases:** `Module`

Patch embedding for MoonViT3d.

**`patch_size`**

---

**`pos_emb`**

---

**`proj`**

---

```python
nemo_automodel.components.models.kimi_k25_vl.model.MoonVision3dPatchEmbed.forward(
    x: torch.Tensor,
    grid_thws: torch.Tensor
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.kimi_k25_vl.model.Rope2DPosEmbRepeated(
    dim: int,
    max_height: int,
    max_width: int,
    theta_base: float = 10000
)
```

**Bases:** `Module`

2D rotary position embedding repeated for temporal dimension.

```python
nemo_automodel.components.models.kimi_k25_vl.model.Rope2DPosEmbRepeated._precompute_freqs_cis(
    device: torch.device
) -> torch.Tensor
```

```python
nemo_automodel.components.models.kimi_k25_vl.model.Rope2DPosEmbRepeated.get_freqs_cis(
    grid_thws: torch.Tensor,
    device: torch.device
) -> torch.Tensor
```

```python
nemo_automodel.components.models.kimi_k25_vl.model._apply_rope_vision(
    xq: torch.Tensor,
    xk: torch.Tensor,
    freqs_cis: torch.Tensor
) -> typing.Tuple[torch.Tensor, torch.Tensor]
```

Apply rotary position embedding for vision.

```python
nemo_automodel.components.models.kimi_k25_vl.model._register_kimi_k25_vl_with_transformers()
```

Register KimiK25VLConfig and model with transformers Auto classes.

```python
nemo_automodel.components.models.kimi_k25_vl.model.compute_expanded_seq_length(
    text_seq_length: int,
    grid_thws: torch.Tensor,
    merge_kernel_size: typing.Tuple[int, int] = (2, 2),
    num_images: int = 1
) -> int
```

Compute the expanded sequence length after image token insertion.

For pipeline parallelism, this can be used to pre-compute the target\_seq\_length
parameter needed for fixed-shape outputs.

**Parameters:**

**`text_seq_length`** `int`

Original text sequence length (including 1 placeholder per image)

---

**`grid_thws`** `torch.Tensor`

Tensor of shape (num\_images, 3) with \[t, h, w] per image

---

**`merge_kernel_size`** `Tuple[int, int]` — default: (2, 2)

Vision tower's patch merge kernel size, default (2, 2)

---

**`num_images`** `int` — default: 1

Number of images (placeholders) in the sequence

---

**Returns:** `int`

Expected sequence length after image features are inserted

```python
nemo_automodel.components.models.kimi_k25_vl.model.get_1d_sincos_pos_embed(
    embed_dim: int,
    t_size: int,
    cls_token: bool = False
) -> numpy.ndarray
```

Generate 1D sinusoidal positional embedding for temporal dimension.

```python
nemo_automodel.components.models.kimi_k25_vl.model.get_1d_sincos_pos_embed_from_grid(
    embed_dim: int,
    pos: numpy.ndarray
) -> numpy.ndarray
```

Generate 1D sinusoidal positional embedding from grid positions.

```python
nemo_automodel.components.models.kimi_k25_vl.model.tpool_patch_merger(
    x: torch.Tensor,
    grid_thws: torch.Tensor,
    merge_kernel_size: typing.List[int] | None = None
) -> typing.List[torch.Tensor]
```

Merge patches with temporal pooling.

```python
nemo_automodel.components.models.kimi_k25_vl.model.vision_attention_flash(
    q,
    k,
    v,
    q_cu_seqlens,
    k_cu_seqlens,
    max_seqlen_q = None,
    max_seqlen_k = None
)
```

Flash attention for vision.

```python
nemo_automodel.components.models.kimi_k25_vl.model.vision_attention_sdpa(
    q,
    k,
    v,
    q_cu_seqlens,
    k_cu_seqlens,
    kwargs = {}
)
```

SDPA attention for vision.

```python
nemo_automodel.components.models.kimi_k25_vl.model.FLASH_ATTN_AVAILABLE = True
```

```python
nemo_automodel.components.models.kimi_k25_vl.model.LOGGER = logging.getLogger(__name__)
```

```python
nemo_automodel.components.models.kimi_k25_vl.model.ModelClass = KimiK25VLForConditionalGeneration
```