ReferenceFull Library ReferenceNemo AutomodelNemo AutomodelComponentsModelsDeepseek V41

nemo_automodel.components.models.deepseek_v41.processing

View as Markdown

DeepSeek-V4.1 image sizing, row-major token spans, and standard chat processor.

Module Contents

Classes

NameDescription
DeepseekV41ImageInputOne image’s patches and placement in a padded text batch.
DeepseekV41ProcessorProcessor for V4.1 text and local image SFT, using the released chat mode.
_ImageGrid-
_ImageRecord-

Functions

NameDescription
_load_imageLoad the local/PIL/byte image forms emitted by the VLM datasets.
_plan_image_gridApply the released aspect-preserving resize and exact row-major token budget.
_preprocess_imageLoad, resize, normalize, and flatten one image into RGB patches.
image_inputs_from_batchValidate and partition processor tensors into ordered image inputs.

Data

IMAGE_PLACEHOLDER

TEXT

_ASSISTANT

_BOS

_EOS

_LABEL_CHAT_TEMPLATE

_SYSTEM

_USER

API

class nemo_automodel.components.models.deepseek_v41.processing.DeepseekV41ImageInput(
    batch_index: int,
    start: int,
    patches: torch.Tensor,
    n_vit_h: int,
    n_vit_w: int,
    types: torch.Tensor
)
Dataclass

One image’s patches and placement in a padded text batch.

batch_index
int
n_vit_h
int
n_vit_w
int
patches
Tensor
start
int
types
Tensor
class nemo_automodel.components.models.deepseek_v41.processing.DeepseekV41Processor(
    tokenizer: transformers.PreTrainedTokenizerFast,
    config: nemo_automodel.components.models.deepseek_v41.config.DeepseekV41Config
)

Bases: ProcessorMixin

Processor for V4.1 text and local image SFT, using the released chat mode.

Tool schemas/calls, reasoning traces, and internal task formatting must be encoded with DeepSeek’s full encoder before calling this processor on text. apply_chat_template rejects those fields instead of silently losing them.

Parameters:

tokenizer
PreTrainedTokenizerFast

Fast tokenizer containing the configured image placeholder.

config
DeepseekV41Config

Typed checkpoint configuration with image and vision settings.

attributes
= ['tokenizer']
tokenizer_class
= 'AutoTokenizer'
nemo_automodel.components.models.deepseek_v41.processing.DeepseekV41Processor.__call__(
    text: str | list[str],
    images: Image.Image | list[Image.Image] | list[list[Image.Image]] | None = None,
    return_tensors: str | None = None,
    padding: bool | str = False,
    truncation: bool = False,
    max_length: int | None = None,
    kwargs: typing.Any = {}
) -> transformers.feature_extraction_utils.BatchFeature

Expand image placeholders and concatenate patches without pseudo token IDs.

Parameters:

text
str | list[str]

Rendered prompt or prompts containing one image placeholder per image.

images
Image.Image | list[Image.Image] | list[list[Image.Image]] | NoneDefaults to None

PIL images grouped per prompt, or one prompt’s flat image list.

return_tensors
str | NoneDefaults to None

"pt" for PyTorch token tensors or None for token lists.

padding
bool | strDefaults to False

False, True/"longest", or "max_length"; padding is on the right.

truncation
boolDefaults to False

Whether max_length removes trailing text and complete trailing images.

max_length
int | NoneDefaults to None

Token limit after image expansion. Cutting through an image raises.

**kwargs
AnyDefaults to {}

Hugging Face tokenizer keyword arguments.

Returns: BatchFeature

BatchFeature with input_ids/attention_mask/vision_token_types of shape

nemo_automodel.components.models.deepseek_v41.processing.DeepseekV41Processor._render_chat(
    messages: collections.abc.Sequence[dict[str, typing.Any]],
    add_generation_prompt: bool
) -> tuple[str, list[Image.Image]]

Render the standard-chat subset of DeepSeek’s released encoding.py.

nemo_automodel.components.models.deepseek_v41.processing.DeepseekV41Processor.apply_chat_template(
    conversation: collections.abc.Sequence[dict[str, typing.Any]] | collections.abc.Sequence[collections.abc.Sequence[dict[str, typing.Any]]],
    tokenize: bool = False,
    return_dict: bool = False,
    return_tensors: str | None = None,
    add_generation_prompt: bool = True,
    processor_kwargs: dict[str, typing.Any] | None = None,
    kwargs: typing.Any = {}
) -> str | list[str] | list[list[int]] | torch.Tensor | transformers.feature_extraction_utils.BatchFeature

Render standard chat-mode system/user/assistant messages and local images.

Parameters:

conversation
Sequence[dict[str, Any]] | Sequence[Sequence[dict[str, Any]]]

Hugging Face chat messages or batches thereof, using string content or ordered text/image content blocks.

tokenize
boolDefaults to False

Whether to expand images and tokenize the rendered prompts.

return_dict
boolDefaults to False

Whether tokenization returns the complete processor batch.

return_tensors
str | NoneDefaults to None

None or "pt" for the tokenization result.

add_generation_prompt
boolDefaults to True

Append the assistant header after a final user or mid-conversation system message, matching official chat mode.

processor_kwargs
dict[str, Any] | NoneDefaults to None

Arguments forwarded to the image/token processor.

**kwargs
AnyDefaults to {}

Hugging Face chat-template options. Non-chat modes and tools raise explicitly; pre-render them with the full official encoder.

Returns: str | list[str] | list[list[int]] | torch.Tensor | BatchFeature

Prompt string(s), token IDs [batch, sequence], or BatchFeature with

classmethod

Load the checkpoint’s fast tokenizer and nested vision configuration.

nemo_automodel.components.models.deepseek_v41.processing.DeepseekV41Processor.save_pretrained(
    save_directory: str | pathlib.Path,
    kwargs: typing.Any = {}
) -> list[str]

Persist the model’s image settings alongside the tokenizer and processor.

Parameters:

save_directory
str | Path

Output directory for the Hugging Face artifacts.

**kwargs
AnyDefaults to {}

Hugging Face ProcessorMixin save options.

Returns: list[str]

Processor artifact paths returned by ProcessorMixin.save_pretrained.

class nemo_automodel.components.models.deepseek_v41.processing._ImageGrid(
    height: int,
    width: int,
    vit_height: int,
    vit_width: int,
    llm_height: int,
    llm_width: int
)
Dataclass
height
int
llm_height
int
llm_width
int
vit_height
int
vit_width
int
width
int
class nemo_automodel.components.models.deepseek_v41.processing._ImageRecord

Bases: typing.TypedDict

bytes
bytes
image
Image
path
str
nemo_automodel.components.models.deepseek_v41.processing._load_image(
    value: Image.Image | str | pathlib.Path | bytes | nemo_automodel.components.models.deepseek_v41.processing._ImageRecord
) -> Image.Image

Load the local/PIL/byte image forms emitted by the VLM datasets.

nemo_automodel.components.models.deepseek_v41.processing._plan_image_grid(
    width: int,
    height: int,
    config: nemo_automodel.components.models.deepseek_v41.config.DeepseekV41VisionConfig
) -> nemo_automodel.components.models.deepseek_v41.processing._ImageGrid

Apply the released aspect-preserving resize and exact row-major token budget.

nemo_automodel.components.models.deepseek_v41.processing._preprocess_image(
    value: Image.Image | str | pathlib.Path | bytes | nemo_automodel.components.models.deepseek_v41.processing._ImageRecord,
    config: nemo_automodel.components.models.deepseek_v41.config.DeepseekV41VisionConfig
) -> tuple[torch.Tensor, nemo_automodel.components.models.deepseek_v41.processing._ImageGrid]

Load, resize, normalize, and flatten one image into RGB patches.

Returns: torch.Tensor

BF16 patches of shape [vit_height * vit_width, 3, patch_size, patch_size]

nemo_automodel.components.models.deepseek_v41.processing.image_inputs_from_batch(
    pixel_values: torch.Tensor,
    image_grid_hws: torch.Tensor,
    vision_token_types: torch.Tensor,
    downsample_ratio: int
) -> tuple[nemo_automodel.components.models.deepseek_v41.processing.DeepseekV41ImageInput, ...]

Validate and partition processor tensors into ordered image inputs.

Parameters:

pixel_values
torch.Tensor

Tensor of shape [all_patches, 3, patch_size, patch_size], concatenated in batch-row then image-span order.

image_grid_hws
torch.Tensor

Integer tensor of shape [images, 2] storing ViT height and width for each image in the same order.

vision_token_types
torch.Tensor

Integer tensor of shape [batch, sequence], with TEXT=-1 outside complete image spans and types 0 through 3 inside.

downsample_ratio
int

Configured spatial ratio mapping the ViT grid to the language model’s image rows and columns.

Returns: DeepseekV41ImageInput

Ordered image records. Each patches/types tensor is a read-only view

nemo_automodel.components.models.deepseek_v41.processing.IMAGE_PLACEHOLDER = '<｜deepseek_image｜>'
 
nemo_automodel.components.models.deepseek_v41.processing.TEXT = -1
 
nemo_automodel.components.models.deepseek_v41.processing._ASSISTANT = '<｜Assistant｜></think>'
 
nemo_automodel.components.models.deepseek_v41.processing._BOS = '<｜begin▁of▁sentence｜>'
 
nemo_automodel.components.models.deepseek_v41.processing._EOS = '<｜end▁of▁sentence｜>'
 
nemo_automodel.components.models.deepseek_v41.processing._LABEL_CHAT_TEMPLATE = "{%- if messages %}{{- bos_token }}{%- endif -%}\n{%- for message in messages -%...
 
nemo_automodel.components.models.deepseek_v41.processing._SYSTEM = '<｜System｜>'
 
nemo_automodel.components.models.deepseek_v41.processing._USER = '<｜User｜>'