bridge.recipes.utils.finetune_utils#

Utility functions for finetuning recipes.

Module Contents#

Functions#

default_peft_config

Create default PEFT configuration matching NeMo2 exactly.

default_squad_config

Create default SQuAD dataset configuration for finetuning recipes.

API#

bridge.recipes.utils.finetune_utils.default_peft_config(
peft_scheme: str | megatron.bridge.peft.base.PEFT | None,
) → megatron.bridge.peft.base.PEFT | None#

Create default PEFT configuration matching NeMo2 exactly.

Parameters:

peft_scheme – PEFT scheme - ‘lora’, ‘dora’, PEFT instance, or None for full finetuning

Returns:

PEFT configuration or None for full finetuning

bridge.recipes.utils.finetune_utils.default_squad_config(
seq_length: int,
packed_sequence: bool = False,
) → megatron.bridge.data.builders.hf_dataset.HFDatasetConfig#

Create default SQuAD dataset configuration for finetuning recipes.

Parameters:

  • seq_length – Sequence length for the dataset

  • packed_sequence – Whether to enable packed sequences for training efficiency

Returns:

HFDatasetConfig configured for SQuAD finetuning

.. note::

Uses consistent settings across all finetuning recipes:

  • SQuAD dataset with appropriate dataloader type

  • 10% validation split

  • Seed 5678 (different from pretrain seed 1234)

  • Packed sequences when enabled improve training efficiency