> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.models.nemotron_parse.model

## Module Contents

### Classes

| Name                                                                                                                                    | Description                                                         |
| --------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
| [`NemotronParseConfig`](#nemo_automodel-components-models-nemotron_parse-model-NemotronParseConfig)                                     | Configuration class for NemotronParse model.                        |
| [`NemotronParseDecoder`](#nemo_automodel-components-models-nemotron_parse-model-NemotronParseDecoder)                                   | Transformer decoder consisting of *config.decoder\_layers* layers.  |
| [`NemotronParseEncoderConfig`](#nemo_automodel-components-models-nemotron_parse-model-NemotronParseEncoderConfig)                       | Configuration class for NemotronParse vision encoder (RADIO-based). |
| [`NemotronParseForConditionalGeneration`](#nemo_automodel-components-models-nemotron_parse-model-NemotronParseForConditionalGeneration) | NemotronParse model for conditional generation tasks.               |
| [`NemotronParsePreTrainedModel`](#nemo_automodel-components-models-nemotron_parse-model-NemotronParsePreTrainedModel)                   | Abstract class to handle weights initialization.                    |
| [`NemotronParseTextConfig`](#nemo_automodel-components-models-nemotron_parse-model-NemotronParseTextConfig)                             | Configuration class for NemotronParse text decoder (mBART-based).   |
| [`RadioWithNeck`](#nemo_automodel-components-models-nemotron_parse-model-RadioWithNeck)                                                 | Vision encoder using RADIO model with custom neck.                  |

### Data

[`ModelClass`](#nemo_automodel-components-models-nemotron_parse-model-ModelClass)

### API

```python
class nemo_automodel.components.models.nemotron_parse.model.NemotronParseConfig(
    encoder: dict | None = None,
    decoder: dict | None = None,
    tie_word_embeddings: bool = False,
    decoder_start_token_id: int = 2,
    pad_token_id: int = 1,
    eos_token_id: int = 2,
    bos_token_id: int = 0,
    image_size: typing.List[int] = None,
    is_encoder_decoder: bool = True,
    max_sequence_length: int = 9000,
    kwargs = {}
)
```

**Bases:** `PretrainedConfig`

Configuration class for NemotronParse model.

**`decoder`** `= NemotronParseTextConfig(**decoder)`

---

**`encoder`**

---

**`image_size`** `= image_size or [2048, 1648]`

---

**`model_type`** `= 'nemotron_parse'`

---

**`vocab_size`** `= self.decoder.vocab_size`

---

```python
nemo_automodel.components.models.nemotron_parse.model.NemotronParseConfig.to_dict()
```

```python
class nemo_automodel.components.models.nemotron_parse.model.NemotronParseDecoder(
    config: transformers.models.mbart.modeling_mbart.MBartConfig,
    embed_tokens: torch.nn.Embedding | None = None
)
```

**Bases:** `MBartPreTrainedModel`

Transformer decoder consisting of *config.decoder\_layers* layers.

**`dropout`** `= config.dropout`

---

**`embed_tokens`**

---

**`layer_norm`** `= nn.LayerNorm(config.d_model)`

---

**`layerdrop`** `= config.decoder_layerdrop`

---

**`layernorm_embedding`** `= nn.LayerNorm(config.d_model)`

---

**`layers`**

---

**`padding_idx`** `= config.pad_token_id`

---

```python
nemo_automodel.components.models.nemotron_parse.model.NemotronParseDecoder.forward(
    input_ids: torch.LongTensor | None = None,
    attention_mask: torch.Tensor | None = None,
    encoder_hidden_states: torch.FloatTensor | None = None,
    encoder_attention_mask: torch.LongTensor | None = None,
    past_key_values: typing.Tuple[typing.Tuple[torch.FloatTensor]] | None = None,
    inputs_embeds: torch.FloatTensor | None = None,
    use_cache: bool | None = None,
    output_attentions: bool | None = None,
    output_hidden_states: bool | None = None,
    return_dict: bool | None = None
) -> typing.Union[typing.Tuple, transformers.models.mbart.modeling_mbart.BaseModelOutputWithPastAndCrossAttentions]
```

```python
nemo_automodel.components.models.nemotron_parse.model.NemotronParseDecoder.get_input_embeddings()
```

```python
nemo_automodel.components.models.nemotron_parse.model.NemotronParseDecoder.set_input_embeddings(
    value
)
```

```python
class nemo_automodel.components.models.nemotron_parse.model.NemotronParseEncoderConfig(
    patch_size: int = 16,
    max_resolution: int = 2048,
    preferred_resolution: typing.List[int] = None,
    torch_dtype: str = 'bfloat16',
    kwargs = {}
)
```

**Bases:** `PretrainedConfig`

Configuration class for NemotronParse vision encoder (RADIO-based).

**`model_type`** `= 'nemotron_parse_encoder'`

---

**`preferred_resolution`** `= preferred_resolution or [768, 768]`

---

```python
class nemo_automodel.components.models.nemotron_parse.model.NemotronParseForConditionalGeneration(
    config: nemo_automodel.components.models.nemotron_parse.model.NemotronParseConfig,
    loss_fn = None,
    kwargs = {}
)
```

**Bases:** [HFCheckpointingMixin](/nemo-automodel/nemo_automodel/components/models/common/hf_checkpointing_mixin#nemo_automodel-components-models-common-hf_checkpointing_mixin-HFCheckpointingMixin), [NemotronParsePreTrainedModel](#nemo_automodel-components-models-nemotron_parse-model-NemotronParsePreTrainedModel), `GenerationMixin`

NemotronParse model for conditional generation tasks.

**`class_token_indx_start`** `= getattr(config, 'class_token_start_idx', 50000)`

---

**`decoder`** `= self.decoder.to(torch.bfloat16)`

---

**`encoder`** `= self.encoder.to(torch.bfloat16)`

---

**`lm_head`**

---

**`tie_word_embeddings_support`** `TieSupport = TieSupport.UNTIED_ONLY`

---

```python
nemo_automodel.components.models.nemotron_parse.model.NemotronParseForConditionalGeneration._reorder_cache(
    past_key_values,
    beam_idx
)
```

```python
nemo_automodel.components.models.nemotron_parse.model.NemotronParseForConditionalGeneration.forward(
    pixel_values: torch.FloatTensor | None = None,
    decoder_input_ids: torch.LongTensor | None = None,
    decoder_attention_mask: torch.BoolTensor | None = None,
    encoder_outputs: typing.Tuple[torch.FloatTensor] | None = None,
    past_key_values: typing.Tuple[typing.Tuple[torch.FloatTensor]] | None = None,
    decoder_inputs_embeds: torch.FloatTensor | None = None,
    labels: torch.LongTensor | None = None,
    use_cache: bool | None = None,
    output_attentions: bool | None = None,
    output_hidden_states: bool | None = None,
    return_dict: bool | None = None,
    logits_to_keep: typing.Union[int, torch.Tensor] = 0,
    kwargs = {}
) -> typing.Union[typing.Tuple[torch.FloatTensor], transformers.modeling_outputs.Seq2SeqLMOutput]
```

```python
nemo_automodel.components.models.nemotron_parse.model.NemotronParseForConditionalGeneration.get_decoder()
```

```python
nemo_automodel.components.models.nemotron_parse.model.NemotronParseForConditionalGeneration.get_encoder()
```

```python
nemo_automodel.components.models.nemotron_parse.model.NemotronParseForConditionalGeneration.get_input_embeddings()
```

```python
nemo_automodel.components.models.nemotron_parse.model.NemotronParseForConditionalGeneration.get_output_embeddings()
```

```python
nemo_automodel.components.models.nemotron_parse.model.NemotronParseForConditionalGeneration.prepare_decoder_input_ids_from_labels(
    labels: torch.Tensor
)
```

```python
nemo_automodel.components.models.nemotron_parse.model.NemotronParseForConditionalGeneration.set_input_embeddings(
    value
)
```

```python
nemo_automodel.components.models.nemotron_parse.model.NemotronParseForConditionalGeneration.set_output_embeddings(
    new_embeddings
)
```

```python
class nemo_automodel.components.models.nemotron_parse.model.NemotronParsePreTrainedModel()
```

**Bases:** `PreTrainedModel`

Abstract class to handle weights initialization.

**`_no_split_modules`** `= ['RadioWithNeck', 'MBartDecoder']`

---

**`_skip_keys_device_placement`** `= 'past_key_values'`

---

**`base_model_prefix`** `= 'vision_encoder_decoder'`

---

**`main_input_name`** `= 'pixel_values'`

---

```python
nemo_automodel.components.models.nemotron_parse.model.NemotronParsePreTrainedModel._init_weights(
    module
)
```

```python
class nemo_automodel.components.models.nemotron_parse.model.NemotronParseTextConfig(
    vocab_size: int = 250027,
    d_model: int = 1024,
    encoder_layers: int = 12,
    decoder_layers: int = 12,
    encoder_attention_heads: int = 16,
    decoder_attention_heads: int = 16,
    decoder_ffn_dim: int = 4096,
    encoder_ffn_dim: int = 4096,
    activation_function: str = 'gelu',
    dropout: float = 0.1,
    attention_dropout: float = 0.0,
    activation_dropout: float = 0.0,
    classifier_dropout: float = 0.0,
    init_std: float = 0.02,
    encoder_layerdrop: float = 0.0,
    decoder_layerdrop: float = 0.0,
    scale_embedding: bool = False,
    use_cache: bool = True,
    num_labels: int = 3,
    forced_eos_token_id: int = 2,
    pad_token_id: int = 1,
    bos_token_id: int = 0,
    eos_token_id: int = 2,
    decoder_start_token_id: int = 2,
    add_cross_attention: bool = True,
    is_decoder: bool = True,
    max_sequence_length: int = 9000,
    kwargs = {}
)
```

**Bases:** `PretrainedConfig`

Configuration class for NemotronParse text decoder (mBART-based).

**`hidden_size`** `= self.d_model`

---

**`model_type`** `= 'nemotron_parse_text'`

---

**`num_attention_heads`** `= self.encoder_attention_heads`

---

```python
class nemo_automodel.components.models.nemotron_parse.model.RadioWithNeck(
    config
)
```

**Bases:** `Module`

Vision encoder using RADIO model with custom neck.

**`conv1`** `= nn.Conv1d(1280, last_hidden_state, 1)`

---

**`conv2`**

---

**`image_processor_normalizes`**

---

**`layer_norm1`**

---

**`layer_norm2`**

---

**`layer_norm3`**

---

**`model_encoder`**

---

**`sum_proj`** `= nn.Linear(3840, last_hidden_state)`

---

```python
nemo_automodel.components.models.nemotron_parse.model.RadioWithNeck._externalize_radio_input_conditioner() -> None
```

```python
nemo_automodel.components.models.nemotron_parse.model.RadioWithNeck.forward(
    pixel_values,
    output_attentions = False,
    output_hidden_states = False,
    return_dict = False,
    kwargs = {}
)
```

Encode document images into visual tokens.

**Parameters:**

**`pixel_values`**

Tensor of shape \[batch, channels, height, width]. When the image processor normalizes
inputs, values must already contain that external normalization.

---

**`output_attentions`** — default: False

Accepted for model interface compatibility and currently unused.

---

**`output_hidden_states`** — default: False

Accepted for model interface compatibility and currently unused.

---

**`return_dict`** — default: False

Accepted for model interface compatibility and currently unused.

---

**`**kwargs`** — default: \{}

Additional model arguments accepted for interface compatibility.

---

**Returns:**

A DonutSwinModelOutput whose last\_hidden\_state has shape \[batch, image\_tokens, hidden].

```python
nemo_automodel.components.models.nemotron_parse.model.ModelClass = NemotronParseForConditionalGeneration
```