nemo_automodel.components.models.deepseek_v4.processing

Processor for DeepSeek-V4-Flash-Vision-Exp.

Module Contents

Classes

Name	Description
`DeepseekV4VisionProcessor`	Exact prompt and dynamic-resolution image processor for DSV4 Vision.

Functions

Name	Description
`_load_pil_image`	Load a supported local, byte, dictionary, or PIL image as RGB.
`build_image_block`	Build reference N-layout token types and aligned-image permutation.
`grid_tokens`	Return the aligned LLM grid and N-layout token count for one image.
`preprocess_image`	Transform one image into normalized ViT patches.
`safe_resize`	Fit an image to the reference N-layout visual-token budget.
`solve_resize_ratio`	Solve the reference resize approximation for a visual-token budget.

Data

ASSISTANT_TOKEN

BOS_TOKEN

COMPRESS_PAD_TO

DEEPSEEK_V4_LABEL_CHAT_TEMPLATE

API

class nemo_automodel.components.models.deepseek_v4.processing.DeepseekV4VisionProcessor(
    tokenizer: typing.Any,
    config: nemo_automodel.components.models.deepseek_v4.config.DeepseekV4Config,
    chat_template: str | None = None,
    kwargs: typing.Any = {}
)

Bases: ProcessorMixin

Exact prompt and dynamic-resolution image processor for DSV4 Vision.

attributes

= ['tokenizer']

image_token_id

int

Return the textual placeholder ID that is expanded before padding.

tokenizer_class

= 'AutoTokenizer'

nemo_automodel.components.models.deepseek_v4.processing.DeepseekV4VisionProcessor.__call__(
    text: str | list[str],
    images: typing.Any = None,
    return_tensors: str | None = None,
    kwargs: typing.Any = {}
) -> transformers.feature_extraction_utils.BatchFeature

Tokenize text and expand each image placeholder into pseudo IDs.

Returns a batch whose input_ids, attention_mask, and vision_token_types use layout [batch, sequence]; pixel_values uses [all_patches, 3, patch_size, patch_size] and image_grid_hws uses [all_images, 2].

nemo_automodel.components.models.deepseek_v4.processing.DeepseekV4VisionProcessor._content_text_and_images(
    content: typing.Any
) -> tuple[str, list[typing.Any]]

staticmethod

Render ordered content blocks and collect their image values.

nemo_automodel.components.models.deepseek_v4.processing.DeepseekV4VisionProcessor._render_conversation(
    conversation: typing.Sequence[dict[str, typing.Any]]
) -> tuple[str, list[typing.Any]]

Render one standard conversation into the released chat format.

nemo_automodel.components.models.deepseek_v4.processing.DeepseekV4VisionProcessor.apply_chat_template(
    conversation: typing.Sequence[dict[str, typing.Any]] | typing.Sequence[typing.Sequence[dict[str, typing.Any]]],
    tokenize: bool = False,
    return_dict: bool = False,
    return_tensors: str | None = None,
    processor_kwargs: dict[str, typing.Any] | None = None,
    kwargs: typing.Any = {}
) -> typing.Any

Render or tokenize one conversation or a batch of conversations.

nemo_automodel.components.models.deepseek_v4.processing.DeepseekV4VisionProcessor.batch_decode(
    args: typing.Any = (),
    kwargs: typing.Any = {}
) -> list[str]

Forward batched decoding to the underlying tokenizer.

nemo_automodel.components.models.deepseek_v4.processing.DeepseekV4VisionProcessor.decode(
    args: typing.Any = (),
    kwargs: typing.Any = {}
) -> str

Forward decoding to the underlying tokenizer.

nemo_automodel.components.models.deepseek_v4.processing.DeepseekV4VisionProcessor.from_pretrained(
    pretrained_model_name_or_path: str,
    kwargs: typing.Any = {}
) -> 'DeepseekV4VisionProcessor'

classmethod

Load the tokenizer and vision settings from one checkpoint.

nemo_automodel.components.models.deepseek_v4.processing._load_pil_image(
    value: typing.Any
) -> PIL.Image.Image

Load a supported local, byte, dictionary, or PIL image as RGB.

nemo_automodel.components.models.deepseek_v4.processing.build_image_block(
    n_llm_h: int,
    n_llm_w: int,
    start_pos: int
) -> tuple[torch.Tensor, torch.Tensor]

Build reference N-layout token types and aligned-image permutation.

Returns: torch.Tensor

types with layout [image_block_tokens] and perm with layout

nemo_automodel.components.models.deepseek_v4.processing.grid_tokens(
    best_height: int,
    best_width: int,
    patch_size: int,
    downsample_ratio: int
) -> tuple[int, int, int]

Return the aligned LLM grid and N-layout token count for one image.

nemo_automodel.components.models.deepseek_v4.processing.preprocess_image(
    value: typing.Any,
    config: nemo_automodel.components.models.deepseek_v4.config.DeepseekV4Config
) -> tuple[torch.Tensor, int, int, int, int]

Transform one image into normalized ViT patches.

Returns: torch.Tensor

Patches with layout [n_vit_h * n_vit_w, 3, patch_size, patch_size]

nemo_automodel.components.models.deepseek_v4.processing.safe_resize(
    height: int | float,
    width: int | float,
    best_height: int,
    best_width: int,
    patch_size: int,
    downsample_ratio: int,
    max_n_token: int
) -> tuple[int, int, int, int]

Fit an image to the reference N-layout visual-token budget.

nemo_automodel.components.models.deepseek_v4.processing.solve_resize_ratio(
    height: int | float,
    width: int | float,
    patch_size: int,
    downsample_ratio: int,
    max_n_token: int
) -> tuple[int, int, int, int, int]

Solve the reference resize approximation for a visual-token budget.

nemo_automodel.components.models.deepseek_v4.processing.ASSISTANT_TOKEN = '<｜Assistant｜>'

nemo_automodel.components.models.deepseek_v4.processing.BOS_TOKEN = '<｜begin▁of▁sentence｜>'

nemo_automodel.components.models.deepseek_v4.processing.COMPRESS_PAD_TO = 4

nemo_automodel.components.models.deepseek_v4.processing.DEEPSEEK_V4_LABEL_CHAT_TEMPLATE = "{%- if messages %}{{- bos_token }}{%- endif -%}\n{%- for message in messages -%...

nemo_automodel.components.models.deepseek_v4.processing.EOS_TOKEN = '<｜end▁of▁sentence｜>'

nemo_automodel.components.models.deepseek_v4.processing.IMAGE_PLACEHOLDER = '<｜deepseek_image｜>'

nemo_automodel.components.models.deepseek_v4.processing.THINKING_END_TOKEN = '</think>'

nemo_automodel.components.models.deepseek_v4.processing.USER_TOKEN = '<｜User｜>'

nemo_automodel.components.models.deepseek_v4.processing.__all__ = ['ASSISTANT_TOKEN', 'BOS_TOKEN', 'COMPRESS_PAD_TO', 'DeepseekV4VisionProcessor',...