bridge.models.minicpmo.modeling_minicpmo#

Understanding-stage composition of HF perception and a Megatron decoder.

Module Contents#

Classes#

MiniCPMOPerception

Build only the checkpoint’s perception modules, without a second LLM.

MiniCPMOModel

Expose the standard Megatron model API for MiniCPM-o understanding.

API#

class bridge.models.minicpmo.modeling_minicpmo.MiniCPMOPerception(
model_config: Any,
*,
source: str,
revision: str | None,
)#

Bases: torch.nn.Module

Build only the checkpoint’s perception modules, without a second LLM.

Initialization

forward(
data: dict[str, Any],
embeddings: torch.Tensor,
) → torch.Tensor#

Run native encoders and insert features at each sample’s own bounds.

encode_audio_samples(
facade: Any,
data: dict[str, Any],
*,
batch_size: int,
) → list[list[torch.Tensor]]#

Encode exact-length clips, retaining sample boundaries and gradients.

Native Whisper attention masks index post-convolution positions using pre-convolution lengths. Cropping before the native call avoids attending to padding introduced by other clips in the batch.

class bridge.models.minicpmo.modeling_minicpmo.MiniCPMOModel(
language_model: torch.nn.Module,
perception: bridge.models.minicpmo.modeling_minicpmo.MiniCPMOPerception,
)#

Bases: megatron.core.transformer.module.MegatronModule

Expose the standard Megatron model API for MiniCPM-o understanding.

Perception is composed with the existing Qwen3-compatible GPT decoder. The initial dense path requires PP=CP=1 and sequence parallelism disabled.

Initialization

shared_embedding_or_output_weight() → torch.Tensor#

Expose the decoder’s tied-embedding synchronization hook.

set_input_tensor(input_tensor: Any) → None#

Delegate the standard Megatron pipeline hook to the decoder.

forward(
input_ids: torch.Tensor,
position_ids: torch.Tensor,
attention_mask: torch.Tensor | None = None,
*,
media: dict[str, Any] | None = None,
labels: torch.Tensor | None = None,
packed_seq_params: Any = None,
inference_params: Any = None,
output_processor: Any = None,
output_processor_context: Any = None,
) → Any#

Compose dense per-sample inputs and preserve the decoder’s output API.