nemo_automodel.components.models.inkling.image_processing

Native image preprocessing for Inkling’s hierarchical vision tower.

Module Contents

Classes

Name	Description
`InklingImageProcessor`	Convert images into Inkling’s padded spatiotemporal patch layout.
`InklingImageProcessorKwargs`	Additional Inkling image preprocessing arguments.

Functions

Name	Description
`_divide_to_patches`	Divide a channels-first image into possibly incomplete square patches.

Data

__all__

API

class nemo_automodel.components.models.inkling.image_processing.InklingImageProcessor(
    kwargs: transformers.processing_utils.Unpack[nemo_automodel.components.models.inkling.image_processing.InklingImageProcessorKwargs] = {}
)

Bases: TorchvisionBackend

Convert images into Inkling’s padded spatiotemporal patch layout.

resample

= PILImageResampling.LANCZOS

rescale_image_max_upscaled_long_edge

= 2048

size

= {'height': 40, 'width': 40}

nemo_automodel.components.models.inkling.image_processing.InklingImageProcessor._preprocess(
    images: list[torch.Tensor],
    size: transformers.image_utils.SizeDict,
    do_rescale: bool,
    rescale_factor: float,
    do_normalize: bool,
    image_mean: float | list[float] | None,
    image_std: float | list[float] | None,
    resample: transformers.image_utils.PILImageResampling | int | None,
    rescale_image_frac: float | None,
    rescale_image_max_upscaled_long_edge: int | None,
    return_tensors: str | transformers.utils.TensorType | None,
    kwargs: typing.Any = {}
) -> transformers.image_processing_utils.BatchFeature

Process normalized channels-first image tensors.

Parameters:

images

list[torch.Tensor]

Tensors of shape [channels, height, width].

size

SizeDict

Square patch dimensions.

do_rescale

bool

Whether to multiply pixels by rescale_factor.

rescale_factor

float

Pixel rescaling multiplier.

do_normalize

bool

Whether to normalize channels.

image_mean

float | list[float] | None

Per-channel normalization means.

image_std

float | list[float] | None

Per-channel normalization standard deviations.

resample

PILImageResampling | int | None

Resize interpolation mode.

rescale_image_frac

float | None

Optional multiplier for the long image edge.

rescale_image_max_upscaled_long_edge

int | None

Maximum long edge when upscaling.

return_tensors

str | TensorType | None

Requested output tensor framework.

**kwargs

AnyDefaults to {}

Unused common backend arguments.

Returns: BatchFeature

A batch containing pixel_values with shape

nemo_automodel.components.models.inkling.image_processing.InklingImageProcessor._validate_preprocess_kwargs(
    do_rescale: bool | None = None,
    rescale_factor: float | None = None,
    do_normalize: bool | None = None,
    image_mean: float | tuple[float, ...] | None = None,
    image_std: float | tuple[float, ...] | None = None,
    do_resize: bool | None = None,
    size: transformers.image_utils.SizeDict | None = None,
    do_center_crop: bool | None = None,
    crop_size: transformers.image_utils.SizeDict | None = None,
    resample: transformers.image_utils.PILImageResampling | int | None = None,
    kwargs: typing.Any = {}
) -> None

Validate model-specific image preprocessing settings.

nemo_automodel.components.models.inkling.image_processing.InklingImageProcessor.preprocess(
    images: transformers.image_utils.ImageInput,
    kwargs: transformers.processing_utils.Unpack[transformers.processing_utils.ImagesKwargs] = {}
) -> transformers.image_processing_utils.BatchFeature

Convert image inputs into vision-tower patches.

Parameters:

images

ImageInput

One image or a batch in a supported PIL, NumPy, or tensor layout.

**kwargs

Unpack[ImagesKwargs]Defaults to {}

Standard Transformers image preprocessing overrides.

Returns: BatchFeature

A batch containing pixel_values with shape

class nemo_automodel.components.models.inkling.image_processing.InklingImageProcessorKwargs()

Bases: ImagesKwargs

Additional Inkling image preprocessing arguments.

rescale_image_frac

float | None

rescale_image_max_upscaled_long_edge

int | None

nemo_automodel.components.models.inkling.image_processing._divide_to_patches(
    image: torch.Tensor,
    patch_size: int
) -> list[torch.Tensor]

Divide a channels-first image into possibly incomplete square patches.

Parameters:

image

torch.Tensor

Tensor of shape [channels, height, width].

patch_size

int

Height and width of each patch.

Returns: list[torch.Tensor]

Tensors of shape [channels, patch_height, patch_width] in row-major order.

nemo_automodel.components.models.inkling.image_processing.__all__ = ['InklingImageProcessor']