`bridge.recipes.qwen.qwen2`#

Module Contents#

Classes#

`Qwen2CommonKwargs`	Typed options accepted by Qwen2/2.5 recipe helper functions.
`Qwen2FinetuneKwargs`	Typed options accepted by Qwen2/2.5 finetuning recipe helper functions.

Functions#

`qwen2_500m_pretrain_config`	Return a pre-training config for Qwen2 0.5B.
`qwen2_1p5b_pretrain_config`	Return a pre-training config for Qwen2 1.5B.
`qwen2_7b_pretrain_config`	Return a pre-training config for Qwen2 7B.
`qwen2_72b_pretrain_config`	Return a pre-training config for Qwen2 72B.
`qwen25_500m_pretrain_config`	Return a pre-training config for Qwen2.5 0.5B.
`qwen25_1p5b_pretrain_config`	Return a pre-training config for Qwen2.5 1.5B.
`qwen25_7b_pretrain_config`	Return a pre-training config for Qwen2.5 7B.
`qwen25_14b_pretrain_config`	Return a pre-training config for Qwen2.5 14B.
`qwen25_32b_pretrain_config`	Return a pre-training config for Qwen2.5 32B.
`qwen25_72b_pretrain_config`	Return a pre-training config for Qwen2.5 72B.
`_qwen2_common`	Create a pre-training configuration for Qwen2/Qwen2.5 models using a given HuggingFace path.
`qwen2_500m_finetune_config`	Return a finetuning config for Qwen2 500M.
`qwen2_1p5b_finetune_config`	Return a finetuning config for Qwen2 1.5B.
`qwen2_7b_finetune_config`	Return a finetuning config for Qwen2 7B.
`qwen2_72b_finetune_config`	Return a finetuning config for Qwen2 72B.
`qwen25_500m_finetune_config`	Return a finetuning config for Qwen2.5 500M.
`qwen25_1p5b_finetune_config`	Return a finetuning config for Qwen2.5 1.5B.
`qwen25_7b_finetune_config`	Return a finetuning config for Qwen2.5 7B.
`qwen25_14b_finetune_config`	Return a finetuning config for Qwen2.5 14B.
`qwen25_32b_finetune_config`	Return a finetuning config for Qwen2.5 32B.
`qwen25_72b_finetune_config`	Return a finetuning config for Qwen2.5 72B.
`_qwen2_finetune_common`	Common finetuning configuration for all Qwen2/2.5 models.

API#

class bridge.recipes.qwen.qwen2.Qwen2CommonKwargs#

Bases: typing_extensions.TypedDict

Typed options accepted by Qwen2/2.5 recipe helper functions.

Initialization

Initialize self. See help(type(self)) for accurate signature.

hf_path: str#: None

dir: Optional[str]#: None

name: str#: None

data_paths: Optional[List[str]]#: None

data_args_path: Optional[str]#: None

train_data_path: Optional[List[str]]#: None

valid_data_path: Optional[List[str]]#: None

test_data_path: Optional[List[str]]#: None

per_split_data_args_path: Optional[str]#: None

mock: bool#: None

tensor_model_parallel_size: int#: None

pipeline_model_parallel_size: int#: None

pipeline_dtype: Optional[torch.dtype]#: None

virtual_pipeline_model_parallel_size: Optional[int]#: None

context_parallel_size: int#: None

sequence_parallel: bool#: None

use_megatron_fsdp: bool#: None

check_for_nan_in_grad: bool#: None

train_iters: int#: None

global_batch_size: int#: None

micro_batch_size: int#: None

seq_length: int#: None

lr: float#: None

min_lr: float#: None

lr_warmup_iters: int#: None

lr_decay_iters: Optional[int]#: None

eval_interval: int#: None

save_interval: int#: None

use_null_tokenizer: bool#: None

precision_config: Optional[Union[megatron.bridge.training.mixed_precision.MixedPrecisionConfig, str]]#: None

comm_overlap_config: Optional[megatron.bridge.training.comm_overlap.CommOverlapConfig]#: None

bridge.recipes.qwen.qwen2.qwen2_500m_pretrain_config(

**user_kwargs: typing_extensions.Unpack[bridge.recipes.qwen.qwen2.Qwen2CommonKwargs],

) → megatron.bridge.training.config.ConfigContainer#

Return a pre-training config for Qwen2 0.5B.