bridge.recipes.qwen2_audio.h100.qwen2_audio

Module Contents

Functions

qwen2_audio_7b_sft_1gpu_h100_bf16_config	Return a fine-tuning config for Qwen2-Audio 7B Instruct.
qwen2_audio_7b_peft_1gpu_h100_bf16_config	Return a PEFT config for Qwen2-Audio 7B Instruct.
_qwen2_audio_common	Create a fine-tuning configuration for Qwen2-Audio models.

Data

__all__

API

bridge.recipes.qwen2_audio.h100.qwen2_audio.qwen2_audio_7b_sft_1gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer

Return a fine-tuning config for Qwen2-Audio 7B Instruct.

Default configuration: 1 node, TP=1, PP=1

• Full SFT: LR=5e-6

bridge.recipes.qwen2_audio.h100.qwen2_audio.qwen2_audio_7b_peft_1gpu_h100_bf16_config(

peft_scheme: str | megatron.bridge.peft.base.PEFT = 'lora',

) → megatron.bridge.training.config.ConfigContainer

Return a PEFT config for Qwen2-Audio 7B Instruct.

bridge.recipes.qwen2_audio.h100.qwen2_audio._qwen2_audio_common(

hf_path: str,

output_dir: str | None = None,

name: str = 'qwen2_audio_finetune',

pretrained_checkpoint: Optional[str] = None,

tensor_model_parallel_size: int = 1,

pipeline_model_parallel_size: int = 1,

pipeline_dtype: Optional[torch.dtype] = None,

virtual_pipeline_model_parallel_size: Optional[int] = None,

context_parallel_size: int = 1,

sequence_parallel: bool = False,

train_iters: int = 2000,

global_batch_size: int = 32,

micro_batch_size: int = 1,

seq_length: int = 4096,

lr: float = 0.0003,

min_lr: float = 3e-05,

lr_warmup_iters: int = 5,

lr_decay_iters: Optional[int] = None,

eval_interval: int = 500,

save_interval: int = 200,

precision_config: Optional[Union[megatron.bridge.training.mixed_precision.MixedPrecisionConfig, str]] = 'bf16_mixed',

freeze_language_model: bool = False,

freeze_audio_model: bool = False,

freeze_audio_projection: bool = False,

peft: Optional[Union[str, megatron.bridge.peft.base.PEFT]] = None,

finetune_lr: Optional[float] = None,

source: megatron.bridge.data.builders.HFDatasetSourceConfig | None = None,

validation_source: megatron.bridge.data.builders.HFDatasetSourceConfig | None = None,

test_source: megatron.bridge.data.builders.HFDatasetSourceConfig | None = None,

wandb_project: Optional[str] = None,

wandb_entity: Optional[str] = None,

wandb_exp_name: Optional[str] = None,

) → megatron.bridge.training.config.ConfigContainer

Create a fine-tuning configuration for Qwen2-Audio models.

bridge.recipes.qwen2_audio.h100.qwen2_audio.__all__

[‘qwen2_audio_7b_peft_1gpu_h100_bf16_config’, ‘qwen2_audio_7b_sft_1gpu_h100_bf16_config’]