nemo_automodel.components.models.deepseek_v41.vision
nemo_automodel.components.models.deepseek_v41.vision
DeepSeek vision encoder and spatial merger with independent batched images.
Module Contents
Classes
Functions
API
Bases: DeepseekV4VisionAligner
The released padded spatial merger and GELU projection into text width.
Parameters:
Top-level V4.1 configuration containing vision and text widths.
Merge each image independently, then batch the projection GEMMs.
Parameters:
Concatenated unmerged image features [patches, vision_dim].
Patch rows for a single-image call.
Patch columns for a single-image call.
Integer patch grids [images, 2] in feature order.
Returns: torch.Tensor
Features [sum(ceil(height / ratio) * ceil(width / ratio)), hidden].
Bases: DeepseekV4VisionTransformer
The released full-attention 2D-RoPE ViT, configured by nested V4.1 fields.
Parameters:
Top-level V4.1 configuration containing the vision dimensions.
Encode independent images with one call per FSDP/checkpointed block.
Parameters:
Normalized RGB patches [patches, 3, patch_size, patch_size] or flattened [patches, 3 * patch_size**2], in image order.
Patch rows for the backward-compatible single-image call.
Patch columns for the backward-compatible single-image call.
Integer grids [images, 3], each (frames, height, width). Frames are independent images, including frame-sharder dummies.
Return the pooler-output contract used by frame sharding.
Returns: torch.Tensor | BaseModelOutputWithPooling
Unmerged features [patches, vision_dim], or a ModelOutput containing
Translate the typed V4.1 config into the existing vision implementation’s config.