bridge.models.inkling.inkling2_bridge#

Hugging Face <-> Megatron conversion for Inkling2.

Module Contents#

Classes#

_InterleavedGatedExpertMapping

Fused [E, 2I, H] experts whose gate and up rows alternate: g0, u0, g1, u1, ....

Inkling2Bridge

Megatron Bridge for Inkling2 checkpoints.

Functions#

_replace_text

Write text to a staged file and move it over path, so a reader never sees a partial file.

_vision_config_dict

Return the image encoder settings under the release’s keys.

Data#

API#

bridge.models.inkling.inkling2_bridge._LLM#

‘model.llm’

bridge.models.inkling.inkling2_bridge._VISION_KEYS#

(‘temporal_patch_size’, ‘native_patch_size’, ‘adapter_merge_size’, ‘projection_hidden_dim’, ‘project…

bridge.models.inkling.inkling2_bridge._VISION_KEY_ALIASES#

None

bridge.models.inkling.inkling2_bridge._EXPERT_SUFFIXES#

(‘.mlp.experts.w13_weight’, ‘.mlp.experts.w2_weight’)

bridge.models.inkling.inkling2_bridge._SCALE_SUFFIXES#

(‘_scale’, ‘_amax’, ‘_outer_scale_denominator’)

bridge.models.inkling.inkling2_bridge._FLOAT32_PARAM_SUFFIXES#

(‘.attn_res_query’, ‘.mlp_res_query’, ‘.output_res_query’, ‘.mlp.router.expert_bias’)

bridge.models.inkling.inkling2_bridge._SAFETENSORS_DTYPES#

None

bridge.models.inkling.inkling2_bridge._replace_text(path: pathlib.Path, text: str) → None#

Write text to a staged file and move it over path, so a reader never sees a partial file.

bridge.models.inkling.inkling2_bridge._vision_config_dict(
vision_config: collections.abc.Mapping[str, Any] | transformers.PretrainedConfig | None,
) → dict[str, Any]#

Return the image encoder settings under the release’s keys.

The release stores vision_config as a dict; native Transformers configs hold it as a config object that reads the same keys as attributes and writes two of them under aliases, which a dict read here may use. Keys the encoder does not read, such as the decoder width that Transformers’ classes add, are left out.

class bridge.models.inkling.inkling2_bridge._InterleavedGatedExpertMapping#

Bases: megatron.bridge.models.conversion.param_mapping.FusedGatedExpertMapping

Fused [E, 2I, H] experts whose gate and up rows alternate: g0, u0, g1, u1, ....

local_hf_param_specs(global_param_name: str | None = None) → tuple#

Interleaved rows have no contiguous per-expert gate/up views.

local_mxfp8_params(*args, **kwargs)#

Native MXFP8 transport cannot express the interleaved row layout.

hf_to_megatron(
hf_weights,
megatron_module: torch.nn.Module,
) → torch.Tensor#

Split one expert’s interleaved rows into gate and up before TP distribution.

megatron_to_hf(
megatron_weights,
megatron_module,
) → dict[str, torch.Tensor]#

Interleave gate and up rows back into the HF expert layout.

class bridge.models.inkling.inkling2_bridge.Inkling2Bridge#

Bases: megatron.bridge.models.conversion.model_bridge.MegatronModelBridge

Megatron Bridge for Inkling2 checkpoints.

Routed experts stored as NVFP4 or MXFP8 are imported in BF16 and quantized back against their source scales on export, so an unchanged model exports the source bytes. Other tensors keep their source dtype, including the FP32 AttnRes queries and router biases.

A plain export (weight_dtype=torch.bfloat16, --export-weight-dtype bfloat16) writes the experts in BF16 without their scale tensors, keeps the FP32 tensors in FP32 as hf_quant_config.json requires, and lists the experts there as BF16. That is the model as Bridge trains it, so a fine-tuned model exports without quantizing onto the source scales.

.. rubric:: Example

from megatron.bridge import AutoBridge bridge = AutoBridge.from_hf_pretrained(“thinkingmachines/Inkling-2-Small”) model_config = bridge.get_model_config()

MODEL_CONFIG_CLASS#

None

USE_MODEL_CONFIG_FOR_CONVERSION#

True

ADDITIONAL_FILE_PATTERNS#

[‘hf_quant_config.json’, ‘processor_config.json’, ‘chat_template.jinja’, ‘tiktoken/*’, ‘mtp.safetens…

PLAIN_EXPORT_IGNORED_SOURCE_KEY_SUFFIXES#

None

hf_config_to_model_config_kwargs(
hf_config: transformers.PretrainedConfig,
) → dict[str, Any]#

Map the nested Inkling2 text configuration onto Inkling2ModelConfig kwargs.

classmethod megatron_to_hf_config(
model_config: megatron.bridge.models.inkling.inkling2_config.Inkling2ModelConfig,
) → dict[str, Any]#

Translate a saved Inkling2ModelConfig back into the release’s nested config layout.

This inverts hf_config_to_model_config_kwargs. AutoBridge.from_auto_config projects the result onto the reference config and takes what the Megatron model does not hold from it: the draft model’s mtp_config, model_max_length, dtype, and eos_token_id.

mapping_registry() → megatron.bridge.models.conversion.mapping_registry.MegatronMappingRegistry#

Map each Inkling2 block to its Hybrid attention and MoE layers, the AttnRes queries, and the image encoder.

Decoder block i is Hybrid layer 2 * i (attention) and 2 * i + 1 (MoE).

build_conversion_tasks(
hf_pretrained: Any,
megatron_model: list[torch.nn.Module],
weight_dtype: torch.dtype | None = None,
*,
exclude_module_types: tuple[type[torch.nn.Module], ...] = (),
exclude_param_patterns: tuple[str, ...] = (),
) → list[megatron.bridge.models.conversion.model_bridge.WeightConversionTask]#

Build the conversion tasks, with the FP32 tensors exported in FP32 by a plain export.

A plain export casts each task’s weights to the task’s weight_dtype. The AttnRes queries and router biases are FP32 in the model and in the release, whose hf_quant_config.json lists them as FP32, so their tasks export in FP32 whatever dtype the export requests.

static get_hf_import_param_names(
hf_param: str | dict[str, str],
available_hf_param_names: set[str] | None = None,
) → tuple[str, ...]#

Declare the scale tensors consumed with quantized expert weights.

maybe_modify_loaded_hf_weight(
hf_param: str | dict[str, str],
hf_state_dict: collections.abc.Mapping[str, torch.Tensor],
) → torch.Tensor | dict[str, torch.Tensor] | megatron.bridge.models.inkling.inkling2_quantization.LazyQuantizedExperts#

Read fused quantized experts lazily, one dequantized expert per access.

_accumulate_grouped_export(
task: megatron.bridge.models.conversion.model_bridge.WeightConversionTask,
converted_weights_dict: dict[str, torch.Tensor],
model_config,
grouped_buffers: dict[str, dict[int, torch.Tensor]],
hf_state_dict: collections.abc.Mapping[str, torch.Tensor],
grouped_sources: dict[str, list[str]] | None = None,
) → dict[str, torch.Tensor] | None#

Quantize each completed fused expert tensor back to its source format and scales.

postprocess_hf_export_weights(path: pathlib.Path) → None#

Describe the experts as the export wrote them, in hf_quant_config.json and config.json.

The precision config copied from the source still names the experts NVFP4 or MXFP8 after a plain export writes them in BF16. Runtimes that read it, such as the reference SGLang runtime, check each tensor’s dtype against it, so the experts move to the BF16 group. A default export writes the source dtypes and leaves the file unchanged.

Transformers, from v5.23.0, applies the release’s MXFP8 quantization to an Inkling2 config.json without transformers_version, which marks the release’s own. A default export writes the experts as the source stores them, so its config.json drops the version, as the release’s has none; a plain export’s BF16 experts keep it, which tells Transformers that they are not quantized.