> This page is for version 0.5.0 · 26.06.
> For other versions, use one of these documentation indexes:
> - Nightly (default): https://docs.nvidia.com/nemo/automodel/nightly/llms.txt
> - Latest: https://docs.nvidia.com/nemo/automodel/latest/llms.txt
> - 0.5.0 · 26.06: https://docs.nvidia.com/nemo/automodel/v0.5/llms.txt
> - 0.4.0 · 26.04: https://docs.nvidia.com/nemo/automodel/v0.4/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.models.llava_onevision

## Submodules

* **[`nemo_automodel.components.models.llava_onevision.model`](/nemo-automodel/nemo_automodel/components/models/llava_onevision/model)**
* **[`nemo_automodel.components.models.llava_onevision.rice_vit`](/nemo-automodel/nemo_automodel/components/models/llava_onevision/rice_vit)**
* **[`nemo_automodel.components.models.llava_onevision.state_dict_adapter`](/nemo-automodel/nemo_automodel/components/models/llava_onevision/state_dict_adapter)**

## Package Contents

### Classes

| Name                                                                                                                                               | Description                                                             |
| -------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
| [`LLaVAOneVision1_5_ForConditionalGeneration`](#nemo_automodel-components-models-llava_onevision-model-LLaVAOneVision1_5_ForConditionalGeneration) | LLaVA-OneVision-1.5 for conditional generation (Rice ViT + Qwen3 text). |
| [`LLaVAOneVision1_5_Model`](#nemo_automodel-components-models-llava_onevision-model-LLaVAOneVision1_5_Model)                                       | Combined vision + language backbone. Returns last\_hidden\_state.       |
| [`LlavaOneVisionStateDictAdapter`](#nemo_automodel-components-models-llava_onevision-state_dict_adapter-LlavaOneVisionStateDictAdapter)            | -                                                                       |
| [`Llavaonevision1_5Config`](#nemo_automodel-components-models-llava_onevision-model-Llavaonevision1_5Config)                                       | Top-level config for LLaVA-OneVision-1.5.                               |
| [`RiceConfig`](#nemo_automodel-components-models-llava_onevision-model-RiceConfig)                                                                 | Configuration for the Rice ViT vision tower.                            |

### API

```python
class nemo_automodel.components.models.llava_onevision.model.LLaVAOneVision1_5_ForConditionalGeneration(
    config,
    attn_implementation: str | None = None,
    kwargs = {}
)
```

**Bases:** [HFCheckpointingMixin](/nemo-automodel/nemo_automodel/components/models/common/hf_checkpointing_mixin#nemo_automodel-components-models-common-hf_checkpointing_mixin-HFCheckpointingMixin), `Module`

LLaVA-OneVision-1.5 for conditional generation (Rice ViT + Qwen3 text).

**`image_token_id`** `= getattr(config, 'image_token_id', 151655)`

---

**`lm_head`**

---

**`model`**

---

**`state_dict_adapter`** `= LlavaOneVisionStateDictAdapter(config)`

---

**`tie_word_embeddings_support`** `TieSupport = TieSupport.UNTIED_ONLY`

---

**`video_token_id`** `= getattr(config, 'video_token_id', 151656)`

---

**`vocab_size`** `= self.model.text_config.vocab_size`

---

```python
nemo_automodel.components.models.llava_onevision.model.LLaVAOneVision1_5_ForConditionalGeneration.forward(
    input_ids: torch.LongTensor | None = None,
    attention_mask: torch.Tensor | None = None,
    position_ids: torch.LongTensor | None = None,
    past_key_values: typing.List[torch.FloatTensor] | None = None,
    inputs_embeds: torch.FloatTensor | None = None,
    labels: torch.LongTensor | None = None,
    use_cache: bool | None = None,
    pixel_values: torch.FloatTensor | None = None,
    pixel_values_videos: torch.FloatTensor | None = None,
    image_grid_thw: torch.LongTensor | None = None,
    video_grid_thw: torch.LongTensor | None = None,
    output_hidden_states: bool | None = None,
    logits_to_keep: typing.Union[int, torch.Tensor] = 0,
    kwargs = {}
) -> typing.Union[typing.Tuple, transformers.modeling_outputs.CausalLMOutputWithPast]
```

```python
nemo_automodel.components.models.llava_onevision.model.LLaVAOneVision1_5_ForConditionalGeneration.from_config(
    config,
    kwargs = {}
)
```

classmethod

```python
nemo_automodel.components.models.llava_onevision.model.LLaVAOneVision1_5_ForConditionalGeneration.from_pretrained(
    pretrained_model_name_or_path: str,
    model_args = (),
    kwargs = {}
)
```

classmethod

```python
nemo_automodel.components.models.llava_onevision.model.LLaVAOneVision1_5_ForConditionalGeneration.get_input_embeddings()
```

```python
nemo_automodel.components.models.llava_onevision.model.LLaVAOneVision1_5_ForConditionalGeneration.get_output_embeddings()
```

```python
nemo_automodel.components.models.llava_onevision.model.LLaVAOneVision1_5_ForConditionalGeneration.set_input_embeddings(
    value
)
```

```python
nemo_automodel.components.models.llava_onevision.model.LLaVAOneVision1_5_ForConditionalGeneration.set_output_embeddings(
    new_embeddings
)
```

```python
class nemo_automodel.components.models.llava_onevision.model.LLaVAOneVision1_5_Model(
    config: nemo_automodel.components.models.llava_onevision.model.Llavaonevision1_5Config,
    attn_implementation: str = 'eager'
)
```

**Bases:** `Module`

Combined vision + language backbone. Returns last\_hidden\_state.

**`language_model`** `= Qwen3Model(self.text_config)`

---

**`text_config`** `= _coerce_text_config(config.text_config)`

---

**`vision_config`** `= _coerce_vision_config(config.vision_config)`

---

**`visual`**

---

```python
nemo_automodel.components.models.llava_onevision.model.LLaVAOneVision1_5_Model.forward(
    input_ids: torch.LongTensor | None = None,
    attention_mask: torch.Tensor | None = None,
    position_ids: torch.LongTensor | None = None,
    past_key_values: typing.List[torch.FloatTensor] | None = None,
    inputs_embeds: torch.FloatTensor | None = None,
    pixel_values: torch.FloatTensor | None = None,
    pixel_values_videos: torch.FloatTensor | None = None,
    image_grid_thw: torch.LongTensor | None = None,
    video_grid_thw: torch.LongTensor | None = None,
    use_cache: bool | None = None,
    kwargs = {}
)
```

```python
nemo_automodel.components.models.llava_onevision.model.LLaVAOneVision1_5_Model.get_image_features(
    pixel_values: torch.FloatTensor,
    image_grid_thw: torch.LongTensor
) -> torch.Tensor
```

```python
nemo_automodel.components.models.llava_onevision.model.LLaVAOneVision1_5_Model.get_input_embeddings()
```

```python
nemo_automodel.components.models.llava_onevision.model.LLaVAOneVision1_5_Model.set_input_embeddings(
    value
)
```

```python
class nemo_automodel.components.models.llava_onevision.state_dict_adapter.LlavaOneVisionStateDictAdapter(
    config: typing.Any = None,
    kwargs = {}
)
```

**Bases:** [StateDictAdapter](/nemo-automodel/nemo_automodel/components/checkpoint/state_dict_adapter#nemo_automodel-components-checkpoint-state_dict_adapter-StateDictAdapter)

```python
nemo_automodel.components.models.llava_onevision.state_dict_adapter.LlavaOneVisionStateDictAdapter.convert_single_tensor_to_hf(
    fqn: str,
    tensor: typing.Any,
    kwargs = {}
) -> list[tuple[str, typing.Any]]
```

```python
nemo_automodel.components.models.llava_onevision.state_dict_adapter.LlavaOneVisionStateDictAdapter.from_hf(
    hf_state_dict: dict[str, typing.Any],
    kwargs = {}
) -> dict[str, typing.Any]
```

```python
nemo_automodel.components.models.llava_onevision.state_dict_adapter.LlavaOneVisionStateDictAdapter.to_hf(
    state_dict: dict[str, typing.Any],
    exclude_key_regex: str | None = None,
    kwargs = {}
) -> dict[str, typing.Any]
```

```python
class nemo_automodel.components.models.llava_onevision.model.Llavaonevision1_5Config(
    text_config: typing.Union[typing.Dict, transformers.configuration_utils.PretrainedConfig] | None = None,
    vision_config: typing.Union[typing.Dict, nemo_automodel.components.models.llava_onevision.model.RiceConfig] | None = None,
    image_token_id: int = 151655,
    video_token_id: int = 151656,
    vision_start_token_id: int = 151652,
    vision_end_token_id: int = 151653,
    vocab_size: int = 152064,
    architectures: typing.List[str] | None = None,
    kwargs = {}
)
```

**Bases:** `PretrainedConfig`

Top-level config for LLaVA-OneVision-1.5.

`model_type` matches the on-hub value exactly so `AutoConfig.from_pretrained`
resolves to this class without `trust_remote_code` once registered.

**`model_type`** `= 'llavaonevision1_5'`

---

**`sub_configs`** `= {'vision_config': RiceConfig}`

---

**`text_config`** `= _build_text_config(text_config)`

---

**`vision_config`** `= RiceConfig(**vision_config)`

---

```python
nemo_automodel.components.models.llava_onevision.model.Llavaonevision1_5Config.to_dict() -> typing.Dict[str, typing.Any]
```

```python
class nemo_automodel.components.models.llava_onevision.model.RiceConfig(
    depth: int = 24,
    embed_dim: int = 1024,
    hidden_size: int = 1024,
    hidden_act: str = 'gelu',
    intermediate_size: int = 4096,
    num_heads: int = 16,
    in_channels: int = 3,
    patch_size: int = 14,
    spatial_merge_size: int = 2,
    temporal_patch_size: int = 1,
    initializer_range: float = 0.02,
    layer_norm_eps: float = 1e-05,
    text_hidden_size: int = 2560,
    kwargs = {}
)
```

**Bases:** `PretrainedConfig`

Configuration for the Rice ViT vision tower.

**`base_config_key`** `= 'vision_config'`

---

**`model_type`** `= 'rice_vit'`

---