bridge.models.inkling.modeling_inkling2.model#
Inkling2 HybridModel: normalized token and image embeddings, AttnRes decoder blocks, and a soft-capped head.
Module Contents#
Classes#
Soft-cap the logits and exclude the padded vocabulary rows from the softmax. |
|
Inkling2 decoder as an MCore HybridModel with an embedding RMSNorm and an optional image encoder. |
Functions#
Reject an attention mask that keeps documents apart, which Inkling2 would ignore. |
API#
- bridge.models.inkling.modeling_inkling2.model.reject_document_masks(
- attention_mask: torch.Tensor | None,
- packed_seq_params: Any,
- cp_group: torch.distributed.ProcessGroup | None = None,
Reject an attention mask that keeps documents apart, which Inkling2 would ignore.
Inkling2βs attention and short convolutions are causal and separate documents only through packed sequences (
packed_seq_params); they never readattention_mask. That loses nothing for a causal mask or for padding at the end, but a mask that hides the first token from a later one, asreset_attention_maskand left padding produce, would be ignored. Context parallelism gives each rank different query rows, so the ranks agree on the verdict overcp_group.- Parameters:
attention_mask β Megatron-Core mask. In its query-by-key form,
[b, 1, s_q, s_k], True hides a key; a[b, s]padding mask is not checked.packed_seq_params β Document boundaries of a packed batch, or None.
cp_group β Context-parallel group, whose ranks hold different query rows.
- Raises:
ValueError β If an unpacked batchβs mask hides the first token from a later one.
- class bridge.models.inkling.modeling_inkling2.model.Inkling2OutputLayerMixin#
Bases:
torch.nn.ModuleSoft-cap the logits and exclude the padded vocabulary rows from the softmax.
- forward(
- *args: Any,
- **kwargs: Any,
- class bridge.models.inkling.modeling_inkling2.model.Inkling2Model(
- config,
- *,
- vocab_size: int,
- pg_collection: megatron.core.process_groups_config.ProcessGroupCollection,
- pre_process: bool,
- post_process: bool,
Bases:
megatron.core.models.hybrid.hybrid_model.HybridModelInkling2 decoder as an MCore HybridModel with an embedding RMSNorm and an optional image encoder.
The Hybrid pattern repeats
*E: each decoder block is an attention layer followed by a MoE layer. The image encoder sits on the embedding stage.Initialization
- _freeze_requested_modules() None#
- embed(
- input_ids: torch.Tensor,
- pixel_values: torch.Tensor | None = None,
- image_grid_thw: torch.Tensor | None = None,
Return full-sequence decoder inputs
[S, B, H]: normalized token embeddings with image features.Image features replace the placeholder positions after the embedding norm, as in the reference implementation.
- forward(
- input_ids: torch.Tensor | None = None,
- position_ids: torch.Tensor | None = None,
- attention_mask: torch.Tensor | None = None,
- *,
- pixel_values: torch.Tensor | None = None,
- image_grid_thw: torch.Tensor | None = None,
- labels: torch.Tensor | None = None,
- loss_mask: torch.Tensor | None = None,
- packed_seq_params: Any = None,
- decoder_input: torch.Tensor | None = None,
- **kwargs: Any,
Embed tokens and images on the first stage, then run the decoder and output head.