bridge.recipes.exaone.h100.exaone_moe

K-EXAONE MoE H100 training recipes.

Module Contents

Functions

get_k_exaone_2_0_pipeline_layout	Build a balanced pipeline layout for K-EXAONE 2.0.
_set_optimizer_precision
_apply_exaone_moe_common
_apply_k_exaone_2_0_topology
exaone_moe_2_0_750b_a37_pretrain_512gpu_h100_bf16_config	Return a pre-training config for K-EXAONE 2.0 750B-A37B MoE.
exaone_moe_2_0_750b_a37_sft_512gpu_h100_bf16_config	Return a full SFT config for K-EXAONE 2.0 750B-A37B MoE.
exaone_moe_2_0_750b_a37_peft_128gpu_h100_bf16_config	Return a PEFT config for K-EXAONE 2.0 750B-A37B MoE.
exaone_moe_236b_a23b_pretrain_64gpu_h100_bf16_config	Return a pre-training config for K-EXAONE 236B-A23B MoE.
exaone_moe_236b_a23b_sft_64gpu_h100_bf16_config	Return a full SFT config for K-EXAONE 236B-A23B MoE.
exaone_moe_236b_a23b_peft_16gpu_h100_bf16_config	Return a PEFT config for K-EXAONE 236B-A23B MoE.

Data

_HF_PATH
_HF_PATH_2_0
_K_EXAONE_2_0_NUM_LAYERS
_K_EXAONE_2_0_MTP_NUM_LAYERS
__all__

API

bridge.recipes.exaone.h100.exaone_moe._HF_PATH

‘LGAI-EXAONE/K-EXAONE-236B-A23B’

bridge.recipes.exaone.h100.exaone_moe._HF_PATH_2_0

‘LGAI-EXAONE/K-EXAONE-2.0-750B-A37B’

bridge.recipes.exaone.h100.exaone_moe._K_EXAONE_2_0_NUM_LAYERS

78

bridge.recipes.exaone.h100.exaone_moe._K_EXAONE_2_0_MTP_NUM_LAYERS

4

bridge.recipes.exaone.h100.exaone_moe.get_k_exaone_2_0_pipeline_layout(

pp_size: int,

vp_size: int | None,

) → list[list[str]] | None

Build a balanced pipeline layout for K-EXAONE 2.0.

bridge.recipes.exaone.h100.exaone_moe._set_optimizer_precision(

cfg: megatron.bridge.training.config.ConfigContainer,

) → None

bridge.recipes.exaone.h100.exaone_moe._apply_exaone_moe_common(

cfg: megatron.bridge.training.config.ConfigContainer,

*,

hf_path: str = _HF_PATH,

) → None

bridge.recipes.exaone.h100.exaone_moe._apply_k_exaone_2_0_topology(

cfg: megatron.bridge.training.config.ConfigContainer,

*,

pipeline_model_parallel_size: int,

expert_model_parallel_size: int,

) → None

bridge.recipes.exaone.h100.exaone_moe.exaone_moe_2_0_750b_a37_pretrain_512gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer

Return a pre-training config for K-EXAONE 2.0 750B-A37B MoE.

bridge.recipes.exaone.h100.exaone_moe.exaone_moe_2_0_750b_a37_sft_512gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer

Return a full SFT config for K-EXAONE 2.0 750B-A37B MoE.

bridge.recipes.exaone.h100.exaone_moe.exaone_moe_2_0_750b_a37_peft_128gpu_h100_bf16_config(

peft_scheme: str | megatron.bridge.peft.base.PEFT = 'lora',

) → megatron.bridge.training.config.ConfigContainer

Return a PEFT config for K-EXAONE 2.0 750B-A37B MoE.

bridge.recipes.exaone.h100.exaone_moe.exaone_moe_236b_a23b_pretrain_64gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer

Return a pre-training config for K-EXAONE 236B-A23B MoE.

bridge.recipes.exaone.h100.exaone_moe.exaone_moe_236b_a23b_sft_64gpu_h100_bf16_config() → megatron.bridge.training.config.ConfigContainer

Return a full SFT config for K-EXAONE 236B-A23B MoE.

bridge.recipes.exaone.h100.exaone_moe.exaone_moe_236b_a23b_peft_16gpu_h100_bf16_config(

peft_scheme: str | megatron.bridge.peft.base.PEFT = 'lora',

) → megatron.bridge.training.config.ConfigContainer

Return a PEFT config for K-EXAONE 236B-A23B MoE.

bridge.recipes.exaone.h100.exaone_moe.__all__

[‘exaone_moe_2_0_750b_a37_peft_128gpu_h100_bf16_config’, ‘exaone_moe_2_0_750b_a37_pretrain_512gpu_h1…