nemo_automodel.components.models.inkling.processing

Processor construction helpers for Inkling fine-tuning.

Module Contents

Functions

Name	Description
`build_inkling_processor`	Load Inkling’s processor and configure padding with an existing token.

Data

_INKLING_END_OF_SAMPLING_TOKEN

API

nemo_automodel.components.models.inkling.processing.build_inkling_processor(
    pretrained_model_name_or_path: str,
    kwargs: typing.Any = {}
) -> transformers.processing_utils.ProcessorMixin

Load Inkling’s processor and configure padding with an existing token.

The published tokenizer does not declare EOS or padding tokens even though its chat template terminates assistant responses with an existing end-of-sampling token. Reusing that token keeps the checkpoint vocabulary unchanged and enables padded fine-tuning batches.

Parameters:

pretrained_model_name_or_path

str

Hugging Face model ID or local snapshot.

**kwargs

AnyDefaults to {}

Additional arguments forwarded to AutoProcessor.from_pretrained.

Returns: ProcessorMixin

The configured Inkling processor.

nemo_automodel.components.models.inkling.processing._INKLING_END_OF_SAMPLING_TOKEN = '<|content_model_end_sampling|>'