> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.models.llava_onevision.rice_vit

Rice Vision Transformer for LLaVA-OneVision-1.5.

Ported from lmms-lab/LLaVA-OneVision-1.5's modeling\_llavaonevision1\_5.py.

## Module Contents

### Classes

| Name                                                                                                    | Description                                                                       |
| ------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- |
| [`RiceAttention`](#nemo_automodel-components-models-llava_onevision-rice_vit-RiceAttention)             | Eager block-diagonal attention over variable-length image segments.               |
| [`RiceBlock`](#nemo_automodel-components-models-llava_onevision-rice_vit-RiceBlock)                     | -                                                                                 |
| [`RiceFlashAttention2`](#nemo_automodel-components-models-llava_onevision-rice_vit-RiceFlashAttention2) | Flash-attention-2 variant using flash\_attn\_varlen\_func (requires flash\_attn). |
| [`RiceMlp`](#nemo_automodel-components-models-llava_onevision-rice_vit-RiceMlp)                         | -                                                                                 |
| [`RicePatchEmbed`](#nemo_automodel-components-models-llava_onevision-rice_vit-RicePatchEmbed)           | -                                                                                 |
| [`RicePatchMerger`](#nemo_automodel-components-models-llava_onevision-rice_vit-RicePatchMerger)         | -                                                                                 |
| [`RiceRotaryEmbedding`](#nemo_automodel-components-models-llava_onevision-rice_vit-RiceRotaryEmbedding) | -                                                                                 |
| [`RiceSdpaAttention`](#nemo_automodel-components-models-llava_onevision-rice_vit-RiceSdpaAttention)     | SDPA variant with an additive block-diagonal mask.                                |
| [`RiceTransformer`](#nemo_automodel-components-models-llava_onevision-rice_vit-RiceTransformer)         | Rice ViT with per-image class-token insertion and block-diagonal attention.       |

### Functions

| Name                                                                                                                    | Description |
| ----------------------------------------------------------------------------------------------------------------------- | ----------- |
| [`apply_rotary_pos_emb_vision`](#nemo_automodel-components-models-llava_onevision-rice_vit-apply_rotary_pos_emb_vision) | -           |
| [`rotate_half`](#nemo_automodel-components-models-llava_onevision-rice_vit-rotate_half)                                 | -           |

### Data

[`_ATTENTION_CLASSES`](#nemo_automodel-components-models-llava_onevision-rice_vit-_ATTENTION_CLASSES)

### API

```python
class nemo_automodel.components.models.llava_onevision.rice_vit.RiceAttention(
    dim: int,
    num_heads: int = 16
)
```

**Bases:** `Module`

Eager block-diagonal attention over variable-length image segments.

```python
nemo_automodel.components.models.llava_onevision.rice_vit.RiceAttention.forward(
    hidden_states: torch.Tensor,
    cu_seqlens: torch.Tensor,
    position_embeddings: typing.Optional[typing.Tuple[torch.Tensor, torch.Tensor]] = None
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.llava_onevision.rice_vit.RiceBlock(
    config,
    attn_implementation: str = 'eager'
)
```

**Bases:** `Module`

```python
nemo_automodel.components.models.llava_onevision.rice_vit.RiceBlock.forward(
    hidden_states: torch.Tensor,
    cu_seqlens: torch.Tensor,
    position_embeddings: typing.Tuple[torch.Tensor, torch.Tensor]
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.llava_onevision.rice_vit.RiceFlashAttention2(
    dim: int,
    num_heads: int = 16
)
```

**Bases:** `Module`

Flash-attention-2 variant using flash\_attn\_varlen\_func (requires flash\_attn).

```python
nemo_automodel.components.models.llava_onevision.rice_vit.RiceFlashAttention2.forward(
    hidden_states: torch.Tensor,
    cu_seqlens: torch.Tensor,
    position_embeddings: typing.Tuple[torch.Tensor, torch.Tensor]
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.llava_onevision.rice_vit.RiceMlp(
    dim: int,
    hidden_dim: int,
    hidden_act: str
)
```

**Bases:** `Module`

```python
nemo_automodel.components.models.llava_onevision.rice_vit.RiceMlp.forward(
    x: torch.Tensor
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.llava_onevision.rice_vit.RicePatchEmbed(
    patch_size: int = 14,
    temporal_patch_size: int = 1,
    in_channels: int = 3,
    embed_dim: int = 1024
)
```

**Bases:** `Module`

```python
nemo_automodel.components.models.llava_onevision.rice_vit.RicePatchEmbed.forward(
    hidden_states: torch.Tensor
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.llava_onevision.rice_vit.RicePatchMerger(
    dim: int,
    context_dim: int,
    spatial_merge_size: int = 2,
    layer_norm_eps: float = 1e-05
)
```

**Bases:** `Module`

```python
nemo_automodel.components.models.llava_onevision.rice_vit.RicePatchMerger.forward(
    x: torch.Tensor
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.llava_onevision.rice_vit.RiceRotaryEmbedding(
    dim: int,
    theta: float = 10000.0
)
```

**Bases:** `Module`

```python
nemo_automodel.components.models.llava_onevision.rice_vit.RiceRotaryEmbedding.forward(
    seqlen: int
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.llava_onevision.rice_vit.RiceSdpaAttention(
    dim: int,
    num_heads: int = 16
)
```

**Bases:** `Module`

SDPA variant with an additive block-diagonal mask.

```python
nemo_automodel.components.models.llava_onevision.rice_vit.RiceSdpaAttention.forward(
    hidden_states: torch.Tensor,
    cu_seqlens: torch.Tensor,
    position_embeddings: typing.Tuple[torch.Tensor, torch.Tensor]
) -> torch.Tensor
```

```python
class nemo_automodel.components.models.llava_onevision.rice_vit.RiceTransformer(
    config,
    attn_implementation: str = 'eager'
)
```

**Bases:** `Module`

Rice ViT with per-image class-token insertion and block-diagonal attention.

Matches the HF reference: one CLS token is prepended at the start of each
image segment inside the flat packed sequence, and the attention mask is
built from a cu\_seqlens that accounts for the extra CLS per segment.

```python
nemo_automodel.components.models.llava_onevision.rice_vit.RiceTransformer.forward(
    pixel_values: torch.Tensor,
    grid_thw: torch.Tensor
) -> torch.Tensor
```

```python
nemo_automodel.components.models.llava_onevision.rice_vit.RiceTransformer.rot_pos_emb(
    grid_thw: torch.Tensor
) -> torch.Tensor
```

```python
nemo_automodel.components.models.llava_onevision.rice_vit.apply_rotary_pos_emb_vision(
    q: torch.Tensor,
    k: torch.Tensor,
    cos: torch.Tensor,
    sin: torch.Tensor
) -> typing.Tuple[torch.Tensor, torch.Tensor]
```

```python
nemo_automodel.components.models.llava_onevision.rice_vit.rotate_half(
    x: torch.Tensor
) -> torch.Tensor
```

```python
nemo_automodel.components.models.llava_onevision.rice_vit._ATTENTION_CLASSES = {'eager': RiceAttention, 'sdpa': RiceSdpaAttention, 'flash_attention_2': RiceFla...
```