bridge.recipes.gemma.gemma2

Module Contents

Functions

gemma2_2b_pretrain_config	Return a pre-training config for Gemma2 2B.
gemma2_9b_pretrain_config	Return a pre-training config for Gemma2 9B.
gemma2_27b_pretrain_config	Return a pre-training config for Gemma2 27B.
_adjust_gemma2_vocab_size	Adjust vocab size for Gemma2 (model vocab < tokenizer vocab).
gemma2_2b_sft_config	Return a full SFT config for Gemma2 2B.
gemma2_9b_sft_config	Return a full SFT config for Gemma2 9B.
gemma2_27b_sft_config	Return a full SFT config for Gemma2 27B.
gemma2_2b_peft_config	Return a PEFT config for Gemma2 2B.
gemma2_9b_peft_config	Return a PEFT config for Gemma2 9B.
gemma2_27b_peft_config	Return a PEFT config for Gemma2 27B.

API

bridge.recipes.gemma.gemma2.gemma2_2b_pretrain_config() → megatron.bridge.training.config.ConfigContainer

Return a pre-training config for Gemma2 2B.

Default parallelism: TP=2, PP=1

bridge.recipes.gemma.gemma2.gemma2_9b_pretrain_config() → megatron.bridge.training.config.ConfigContainer

Return a pre-training config for Gemma2 9B.

Default parallelism: TP=8, PP=1

bridge.recipes.gemma.gemma2.gemma2_27b_pretrain_config() → megatron.bridge.training.config.ConfigContainer

Return a pre-training config for Gemma2 27B.

Default parallelism: TP=8, PP=2

bridge.recipes.gemma.gemma2._adjust_gemma2_vocab_size(model_cfg, hf_path: str)

Adjust vocab size for Gemma2 (model vocab < tokenizer vocab).

Note: This requires HuggingFace authentication for Gemma2 models. If the tokenizer cannot be loaded, the vocab size adjustment is skipped.

bridge.recipes.gemma.gemma2.gemma2_2b_sft_config() → megatron.bridge.training.config.ConfigContainer

Return a full SFT config for Gemma2 2B.

Default parallelism: TP=1, PP=1

Returns:

ConfigContainer with all settings pre-configured for Gemma2 2B SFT.

bridge.recipes.gemma.gemma2.gemma2_9b_sft_config() → megatron.bridge.training.config.ConfigContainer

Return a full SFT config for Gemma2 9B.

Default parallelism: TP=4, PP=1

Returns:

ConfigContainer with all settings pre-configured for Gemma2 9B SFT.

bridge.recipes.gemma.gemma2.gemma2_27b_sft_config() → megatron.bridge.training.config.ConfigContainer

Return a full SFT config for Gemma2 27B.

Default parallelism: TP=8, PP=2

Returns:

ConfigContainer with all settings pre-configured for Gemma2 27B SFT.

bridge.recipes.gemma.gemma2.gemma2_2b_peft_config(

peft_scheme: str | megatron.bridge.peft.base.PEFT = 'lora',

) → megatron.bridge.training.config.ConfigContainer

Return a PEFT config for Gemma2 2B.

Default parallelism: TP=1, PP=1

Parameters:

peft_scheme – PEFT scheme - “lora”, “dora”, or a custom PEFT instance.

Returns:

ConfigContainer with all settings pre-configured for Gemma2 2B PEFT.

bridge.recipes.gemma.gemma2.gemma2_9b_peft_config(

peft_scheme: str | megatron.bridge.peft.base.PEFT = 'lora',

) → megatron.bridge.training.config.ConfigContainer

Return a PEFT config for Gemma2 9B.

Default parallelism: TP=1, PP=1

Parameters:

peft_scheme – PEFT scheme - “lora”, “dora”, or a custom PEFT instance.

Returns:

ConfigContainer with all settings pre-configured for Gemma2 9B PEFT.

bridge.recipes.gemma.gemma2.gemma2_27b_peft_config(

peft_scheme: str | megatron.bridge.peft.base.PEFT = 'lora',

) → megatron.bridge.training.config.ConfigContainer

Return a PEFT config for Gemma2 27B.

Default parallelism: TP=4, PP=1

Parameters:

peft_scheme – PEFT scheme - “lora”, “dora”, or a custom PEFT instance.

Returns:

ConfigContainer with all settings pre-configured for Gemma2 27B PEFT.