nemo_automodel.components.models.inkling2.processing

Released Inkling2 image/chat preprocessing for the shared VLM collator.

Module Contents

Classes

Name	Description
`Inkling2Processor`	Image/text processor with explicit image-body ownership and assistant masks.

Functions

Name	Description
`image_patches`	Prepare one unresized image using the released 40-pixel patch policy.

Data

_CHAT_TEMPLATE

_IMAGE_BODY

API

class nemo_automodel.components.models.inkling2.processing.Inkling2Processor(
    tokenizer: transformers.PreTrainedTokenizerBase,
    chat_template: str | None = None
)

Bases: ProcessorMixin

Image/text processor with explicit image-body ownership and assistant masks.

attributes

= ['tokenizer']

image_body_id

= self.tokenizer.convert_tokens_to_ids(_IMAGE_BODY)

image_token_id

= 200005

tokenizer_class

= 'PreTrainedTokenizerBase'

nemo_automodel.components.models.inkling2.processing.Inkling2Processor.apply_chat_template(
    conversation: list[dict[str, typing.Any]] | list[list[dict[str, typing.Any]]],
    tokenize: bool = False,
    return_dict: bool = False,
    return_tensors: str | None = None,
    return_assistant_tokens_mask: bool = False,
    processor_kwargs: dict[str, typing.Any] | None = None,
    add_generation_prompt: bool = False,
    kwargs: typing.Any = {}
) -> transformers.feature_extraction_utils.BatchFeature | torch.Tensor | str | list[str]

Render image/text conversations through the released control-token grammar.

Parameters:

conversation

list[dict[str, Any]] | list[list[dict[str, Any]]]

One conversation or a batch of HF-style role/content records. Images must be PIL objects; text content is a string or text record.

tokenize

boolDefaults to False

Produce token IDs and patches when true, rendered strings otherwise.

return_dict

boolDefaults to False

Return token fields plus media tensors when true.

return_tensors

str | NoneDefaults to None

Tensor framework; training supports only ‘pt’.

return_assistant_tokens_mask

boolDefaults to False

Include the template’s assistant supervision mask.

processor_kwargs

dict[str, Any] | NoneDefaults to None

HF tokenizer padding, truncation and maximum-length options.

add_generation_prompt

boolDefaults to False

Accepted for the HF interface; this renderer adds no prefix.

kwargs

AnyDefaults to {}

Additional HuggingFace tokenizer template options.

Returns: BatchFeature | torch.Tensor | str | list[str]

HF-compatible token output. Token fields are [batch, sequence]; image patches

nemo_automodel.components.models.inkling2.processing.Inkling2Processor.from_pretrained(
    pretrained_model_name_or_path: str | os.PathLike,
    cache_dir: str | os.PathLike | None = None,
    force_download: bool = False,
    local_files_only: bool = False,
    token: str | bool | None = None,
    revision: str = 'main',
    kwargs: typing.Any = {}
) -> nemo_automodel.components.models.inkling2.processing.Inkling2Processor

classmethod

Load the released tokenizer and verify its image preprocessing contract.

nemo_automodel.components.models.inkling2.processing.image_patches(
    image: PIL.Image.Image
) -> tuple[torch.Tensor, torch.Tensor]

Prepare one unresized image using the released 40-pixel patch policy.

RGBA is composited onto an 8-pixel 204/153 checkerboard. Padding is -1 in pixel space. Normalization uses the release’s FP32 CLIP constants, then casts to BF16 and duplicates each spatial patch along the temporal axis.

Returns: torch.Tensor

BF16 patches [rows*columns, 2, 40, 40, 3] and int64 grid [1, 3].

nemo_automodel.components.models.inkling2.processing._CHAT_TEMPLATE = "{%- for message in messages -%}\n{%- set role = 'model' if message['role'] == '...

nemo_automodel.components.models.inkling2.processing._IMAGE_BODY = '<|unused_200054|>'