nemo_automodel.components.models.pixtral.compat

Temporary Pixtral forward backport for Transformers 5.17.0, 5.18.0, and 5.19.0.

Adapted from Hugging Face Transformers (Apache-2.0), with the image-boundary fix from https://github.com/huggingface/transformers/pull/49373 (ff704ffd47d800e31b31f2f81a5a2952fb0fbf71). Remove this module and its binding when AutoModel pins a release containing that fix.

Module Contents

Functions

Name	Description
`_pixtral_vision_forward`	Preserve axial positions while passing explicit FlashAttention image boundaries.
`apply_pixtral_forward_backport`	Apply HF #49373 to one vision tower on the affected Transformers releases.

API

nemo_automodel.components.models.pixtral.compat._pixtral_vision_forward(
    self: transformers.models.pixtral.modeling_pixtral.PixtralVisionModel,
    pixel_values: torch.Tensor,
    image_sizes: torch.Tensor | None = None,
    kwargs: transformers.processing_utils.Unpack[transformers.utils.TransformersKwargs] = {}
) -> tuple | transformers.modeling_outputs.BaseModelOutput | transformers.modeling_outputs.BaseModelOutputWithPooling

Preserve axial positions while passing explicit FlashAttention image boundaries.

Parameters:

pixel_values

torch.Tensor

Tensor of shape [images, channels, height, width], padded to a common image size.

image_sizes

torch.Tensor | NoneDefaults to None

Tensor of shape [images, 2] containing each unpadded height and width. A list of height/width pairs is also accepted upstream.

**kwargs

Unpack[TransformersKwargs]Defaults to {}

Options following PixtralVisionModel.forward’s contract.

Returns: tuple | BaseModelOutput | BaseModelOutputWithPooling

The PixtralVisionModel.forward output contract: last hidden state

nemo_automodel.components.models.pixtral.compat.apply_pixtral_forward_backport(
    vision_tower: transformers.models.pixtral.modeling_pixtral.PixtralVisionModel
) -> None

Apply HF #49373 to one vision tower on the affected Transformers releases.

Remove this compatibility hook and its callers when the pinned release contains the upstream fix and passes the Pixtral regression tests.

Parameters:

vision_tower

PixtralVisionModel

The owning model’s Pixtral tower. Its forward method is replaced in place; independently constructed HF towers are unchanged.