bridge.recipes.gemma4_vl.gemma4_vl#

Gemma 4 VL finetuning recipes.

This module provides SFT and PEFT configurations for Gemma 4 VL 26B-A4B (MoE VLM).

Module Contents#

Functions#

_apply_gemma4_vl_common

Apply settings common to all Gemma 4 VL 26B-A4B recipes.

gemma4_vl_26b_sft_config

Return a full SFT config for Gemma 4 VL 26B-A4B (MoE VLM).

gemma4_vl_26b_peft_config

Return a PEFT (LoRA/DoRA) config for Gemma 4 VL 26B-A4B (MoE VLM).

Data#

_HF_PATH

API#

bridge.recipes.gemma4_vl.gemma4_vl._HF_PATH#

‘google/gemma-4-26B-A4B-it’

bridge.recipes.gemma4_vl.gemma4_vl._apply_gemma4_vl_common(
cfg: megatron.bridge.training.config.ConfigContainer,
hf_path: str,
) → None#

Apply settings common to all Gemma 4 VL 26B-A4B recipes.

bridge.recipes.gemma4_vl.gemma4_vl.gemma4_vl_26b_sft_config(
hf_path: str = _HF_PATH,
) → megatron.bridge.training.config.ConfigContainer#

Return a full SFT config for Gemma 4 VL 26B-A4B (MoE VLM).

Default configuration: 2 nodes, 16 GPUs

  • TP=2, PP=1, EP=8 (max EP with 16 GPUs at TP=2,PP=1; DP=8, EP divides DP)

  • No activation recompute — EP=8 shards 87.5% of expert params per GPU

  • LR=5e-5 (full SFT)

  • Sequence length: 4096

bridge.recipes.gemma4_vl.gemma4_vl.gemma4_vl_26b_peft_config(
peft_scheme: str | megatron.bridge.peft.base.PEFT = 'lora',
hf_path: str = _HF_PATH,
) → megatron.bridge.training.config.ConfigContainer#

Return a PEFT (LoRA/DoRA) config for Gemma 4 VL 26B-A4B (MoE VLM).

Default configuration: 1 node, 8 GPUs

  • TP=4, PP=1, EP=4 (PEFT needs less memory, drop PP)

  • LR=2e-4 (PEFT)

  • Sequence length: 4096

Parameters:

  • peft_scheme – PEFT scheme — “lora”, “dora”, or a custom PEFT instance.

  • hf_path – HuggingFace model ID or local path.