bridge.models.inkling.inkling2_bridge#
Hugging Face <-> Megatron conversion for Inkling2.
Module Contents#
Classes#
Fused |
|
Megatron Bridge for Inkling2 checkpoints. |
Functions#
Write |
|
Return the image encoder settings under the release’s keys. |
Data#
API#
- bridge.models.inkling.inkling2_bridge._LLM#
‘model.llm’
- bridge.models.inkling.inkling2_bridge._VISION_KEYS#
(‘temporal_patch_size’, ‘native_patch_size’, ‘adapter_merge_size’, ‘projection_hidden_dim’, ‘project…
- bridge.models.inkling.inkling2_bridge._VISION_KEY_ALIASES#
None
- bridge.models.inkling.inkling2_bridge._EXPERT_SUFFIXES#
(‘.mlp.experts.w13_weight’, ‘.mlp.experts.w2_weight’)
- bridge.models.inkling.inkling2_bridge._SCALE_SUFFIXES#
(‘_scale’, ‘_amax’, ‘_outer_scale_denominator’)
- bridge.models.inkling.inkling2_bridge._FLOAT32_PARAM_SUFFIXES#
(‘.attn_res_query’, ‘.mlp_res_query’, ‘.output_res_query’, ‘.mlp.router.expert_bias’)
- bridge.models.inkling.inkling2_bridge._SAFETENSORS_DTYPES#
None
- bridge.models.inkling.inkling2_bridge._replace_text(path: pathlib.Path, text: str) None#
Write
textto a staged file and move it overpath, so a reader never sees a partial file.
- bridge.models.inkling.inkling2_bridge._vision_config_dict(
- vision_config: collections.abc.Mapping[str, Any] | transformers.PretrainedConfig | None,
Return the image encoder settings under the release’s keys.
The release stores
vision_configas a dict; native Transformers configs hold it as a config object that reads the same keys as attributes and writes two of them under aliases, which a dict read here may use. Keys the encoder does not read, such as the decoder width that Transformers’ classes add, are left out.
- class bridge.models.inkling.inkling2_bridge._InterleavedGatedExpertMapping#
Bases:
megatron.bridge.models.conversion.param_mapping.FusedGatedExpertMappingFused
[E, 2I, H]experts whose gate and up rows alternate:g0, u0, g1, u1, ....- local_hf_param_specs(global_param_name: str | None = None) tuple#
Interleaved rows have no contiguous per-expert gate/up views.
- local_mxfp8_params(*args, **kwargs)#
Native MXFP8 transport cannot express the interleaved row layout.
- hf_to_megatron(
- hf_weights,
- megatron_module: torch.nn.Module,
Split one expert’s interleaved rows into gate and up before TP distribution.
- megatron_to_hf(
- megatron_weights,
- megatron_module,
Interleave gate and up rows back into the HF expert layout.
- class bridge.models.inkling.inkling2_bridge.Inkling2Bridge#
Bases:
megatron.bridge.models.conversion.model_bridge.MegatronModelBridgeMegatron Bridge for Inkling2 checkpoints.
Routed experts stored as NVFP4 or MXFP8 are imported in BF16 and quantized back against their source scales on export, so an unchanged model exports the source bytes. Other tensors keep their source dtype, including the FP32 AttnRes queries and router biases.
A plain export (
weight_dtype=torch.bfloat16,--export-weight-dtype bfloat16) writes the experts in BF16 without their scale tensors, keeps the FP32 tensors in FP32 ashf_quant_config.jsonrequires, and lists the experts there as BF16. That is the model as Bridge trains it, so a fine-tuned model exports without quantizing onto the source scales... rubric:: Example
from megatron.bridge import AutoBridge bridge = AutoBridge.from_hf_pretrained(“thinkingmachines/Inkling-2-Small”) model_config = bridge.get_model_config()
- MODEL_CONFIG_CLASS#
None
- USE_MODEL_CONFIG_FOR_CONVERSION#
True
- ADDITIONAL_FILE_PATTERNS#
[‘hf_quant_config.json’, ‘processor_config.json’, ‘chat_template.jinja’, ‘tiktoken/*’, ‘mtp.safetens…
- PLAIN_EXPORT_IGNORED_SOURCE_KEY_SUFFIXES#
None
- hf_config_to_model_config_kwargs(
- hf_config: transformers.PretrainedConfig,
Map the nested Inkling2 text configuration onto
Inkling2ModelConfigkwargs.
- classmethod megatron_to_hf_config(
- model_config: megatron.bridge.models.inkling.inkling2_config.Inkling2ModelConfig,
Translate a saved
Inkling2ModelConfigback into the release’s nested config layout.This inverts
hf_config_to_model_config_kwargs.AutoBridge.from_auto_configprojects the result onto the reference config and takes what the Megatron model does not hold from it: the draft model’smtp_config,model_max_length,dtype, andeos_token_id.
- mapping_registry() megatron.bridge.models.conversion.mapping_registry.MegatronMappingRegistry#
Map each Inkling2 block to its Hybrid attention and MoE layers, the AttnRes queries, and the image encoder.
Decoder block
iis Hybrid layer2 * i(attention) and2 * i + 1(MoE).
- build_conversion_tasks(
- hf_pretrained: Any,
- megatron_model: list[torch.nn.Module],
- weight_dtype: torch.dtype | None = None,
- *,
- exclude_module_types: tuple[type[torch.nn.Module], ...] = (),
- exclude_param_patterns: tuple[str, ...] = (),
Build the conversion tasks, with the FP32 tensors exported in FP32 by a plain export.
A plain export casts each task’s weights to the task’s
weight_dtype. The AttnRes queries and router biases are FP32 in the model and in the release, whosehf_quant_config.jsonlists them as FP32, so their tasks export in FP32 whatever dtype the export requests.
- static get_hf_import_param_names(
- hf_param: str | dict[str, str],
- available_hf_param_names: set[str] | None = None,
Declare the scale tensors consumed with quantized expert weights.
- maybe_modify_loaded_hf_weight(
- hf_param: str | dict[str, str],
- hf_state_dict: collections.abc.Mapping[str, torch.Tensor],
Read fused quantized experts lazily, one dequantized expert per access.
- _accumulate_grouped_export(
- task: megatron.bridge.models.conversion.model_bridge.WeightConversionTask,
- converted_weights_dict: dict[str, torch.Tensor],
- model_config,
- grouped_buffers: dict[str, dict[int, torch.Tensor]],
- hf_state_dict: collections.abc.Mapping[str, torch.Tensor],
- grouped_sources: dict[str, list[str]] | None = None,
Quantize each completed fused expert tensor back to its source format and scales.
- postprocess_hf_export_weights(path: pathlib.Path) None#
Describe the experts as the export wrote them, in
hf_quant_config.jsonandconfig.json.The precision config copied from the source still names the experts NVFP4 or MXFP8 after a plain export writes them in BF16. Runtimes that read it, such as the reference SGLang runtime, check each tensor’s dtype against it, so the experts move to the BF16 group. A default export writes the source dtypes and leaves the file unchanged.
Transformers, from v5.23.0, applies the release’s MXFP8 quantization to an Inkling2
config.jsonwithouttransformers_version, which marks the release’s own. A default export writes the experts as the source stores them, so itsconfig.jsondrops the version, as the release’s has none; a plain export’s BF16 experts keep it, which tells Transformers that they are not quantized.