bridge.models.inkling.inkling2_config#
Builder-backed configuration for the Inkling2 HybridModel integration.
Module Contents#
Classes#
Transformer settings read by the Inkling2 attention, AttnRes, and MoE layers. |
|
Inkling2 HybridModel with an optional image encoder on the embedding stage. |
API#
- class bridge.models.inkling.inkling2_config.Inkling2TransformerConfig#
Bases:
megatron.bridge.models.transformer_config.TransformerConfigTransformer settings read by the Inkling2 attention, AttnRes, and MoE layers.
- variable_seq_lengths: bool#
True
Each layer passes the AttnRes bank packed with its partial block, so the width of the tensor between pipeline stages depends on the boundary and pipeline communication exchanges shapes.
- hetereogenous_dist_checkpoint: bool#
True
Save each layer’s parameters under its own checkpoint key. The layers differ: full-attention layers carry three AttnRes queries and key and value input norms, and the first layer has no attention query, so the parameters cannot be stacked along a common layer axis.
- inkling2_local_layers: tuple[int, ...]#
()
0-based indices of the sliding-window decoder blocks; the remaining blocks use full attention. Block
iis Hybrid layers2 * i(attention) and2 * i + 1(MoE).
- inkling2_sliding_window: int#
1024
Local attention span, including the current token; also the local relative extent.
- inkling2_rel_dim: int#
16
Width of the per-head query feature that produces relative logits.
- inkling2_rel_extent: int#
1024
Number of causal distances with a learned relative logit in full-attention layers.
- inkling2_qk_clip_threshold: float#
22.627416997969522
- inkling2_rel_clip_threshold: float#
8.0
- inkling2_value_norm: bool#
False
Whether values pass through a weightless RMSNorm over the head dimension.
- inkling2_log_scaling_n_floor: float#
128000.0
- inkling2_log_scaling_alpha: float#
0.1
- inkling2_sconv_kernel_size: int#
4
- inkling2_attn_res_block_size: int#
6
Number of decoder blocks whose outputs form one AttnRes block.
- inkling2_attn_res_normalize_values: bool#
False
- inkling2_mlp_gate_softcap: float#
0.0
- inkling2_mlp_up_softcap: float#
0.0
- inkling2_final_logit_softcap: float | None#
None
- inkling2_unpadded_vocab_size: int | None#
None
Logits at and above this token ID are masked out of the softmax.
- inkling2_compile_ops: bool#
False
Run the AttnRes mixtures, short convolutions, L2 clips, SiTU activations, and residual additions through
torch.compileas fused kernels. Dynamo compiles a graph for every new call signature, so this suits training at fixed sequence shapes; leave it off for inference over changing sequence lengths.
- class bridge.models.inkling.inkling2_config.Inkling2ModelConfig#
Bases:
megatron.bridge.models.hybrid.model_config.BridgeHybridModelConfigInkling2 HybridModel with an optional image encoder on the embedding stage.
Each decoder block is an attention layer followed by a MoE layer, so the Hybrid pattern repeats
*Eandnum_layersis twice the block count.- builder: ClassVar[str]#
‘megatron.bridge.models.inkling.inkling2_builder.Inkling2ModelBuilder’
- transformer_config_class: ClassVar[type[megatron.bridge.models.transformer_config.TransformerConfig]]#
None
- vision_config: dict[str, Any]#
‘field(…)’
HF
vision_configof the checkpoint; empty for a text-only model.
- image_token_id: int | None#
None
Placeholder token ID replaced by image features:
<|unused_200054|>(200054) in the release.
- freeze_language_model: bool#
False
- freeze_vision_model: bool#
False
- freeze_vision_projection: bool#
False
- property special_token_ids: dict[str, int]#
Return the media placeholder IDs for collators.
- finalize() None#
Reject execution paths that the Inkling2 layers do not implement.
- _get_num_floating_point_operations_with_runtime_stats(
- *,
- batch_size: int,
- seqlen_sum: int | None,
- seqlen_squared_sum: int | None,
- cross_seqlen_sum: int | None = None,
- cross_seqlen_product_sum: int | None = None,
Return the forward and backward FLOPs of one global batch of text.
Counts the matrix multiplications with the conventions of the standard GPT estimate: the query, key, value, relative-query, gate, sink, and output projections; the dynamic-convolution projections of full-attention layers; the relative logits; causal attention over the full context or the sliding window; the router and the routed experts; and the output layer. Normalization, short convolutions, and the AttnRes mixtures are elementwise or per-source reductions and, like norms in the standard estimate, are not counted; neither is the image encoder.
- Parameters:
batch_size – Number of sequences in the global batch.
seqlen_sum – Total tokens in the global batch, or
Nonefor full-length sequences.seqlen_squared_sum – Sum of squared subsequence lengths, or
Nonefor full-length sequences.cross_seqlen_sum – Unused; Inkling2 has no cross-attention.
cross_seqlen_product_sum – Unused; Inkling2 has no cross-attention.
- Returns:
Floating-point operations of one training step.