bridge.models.minicpmo.modeling_minicpmo#
Understanding-stage composition of HF perception and a Megatron decoder.
Module Contents#
Classes#
Build only the checkpoint’s perception modules, without a second LLM. |
|
Expose the standard Megatron model API for MiniCPM-o understanding. |
API#
- class bridge.models.minicpmo.modeling_minicpmo.MiniCPMOPerception(
- model_config: Any,
- *,
- source: str,
- revision: str | None,
Bases:
torch.nn.ModuleBuild only the checkpoint’s perception modules, without a second LLM.
Initialization
- forward(
- data: dict[str, Any],
- embeddings: torch.Tensor,
Run native encoders and insert features at each sample’s own bounds.
- encode_audio_samples(
- facade: Any,
- data: dict[str, Any],
- *,
- batch_size: int,
Encode exact-length clips, retaining sample boundaries and gradients.
Native Whisper attention masks index post-convolution positions using pre-convolution lengths. Cropping before the native call avoids attending to padding introduced by other clips in the batch.
- class bridge.models.minicpmo.modeling_minicpmo.MiniCPMOModel(
- language_model: torch.nn.Module,
- perception: bridge.models.minicpmo.modeling_minicpmo.MiniCPMOPerception,
Bases:
megatron.core.transformer.module.MegatronModuleExpose the standard Megatron model API for MiniCPM-o understanding.
Perception is composed with the existing Qwen3-compatible GPT decoder. The initial dense path requires PP=CP=1 and sequence parallelism disabled.
Initialization
Expose the decoder’s tied-embedding synchronization hook.
- set_input_tensor(input_tensor: Any) None#
Delegate the standard Megatron pipeline hook to the decoder.
- forward(
- input_ids: torch.Tensor,
- position_ids: torch.Tensor,
- attention_mask: torch.Tensor | None = None,
- *,
- media: dict[str, Any] | None = None,
- labels: torch.Tensor | None = None,
- packed_seq_params: Any = None,
- inference_params: Any = None,
- output_processor: Any = None,
- output_processor_context: Any = None,
Compose dense per-sample inputs and preserve the decoder’s output API.