bridge.recipes.deepseek.deepseek_v4

Hardware-agnostic DeepSeek V4 recipes and compatibility aliases.

Module Contents

Functions

_apply_deepseek_v4_lora	Apply the DeepSeek V4 LoRA convergence contract to an SFT config.
deepseek_v4_flash_sft_openmath_thinking_packed_config	DSv4 Flash SFT on OpenMathInstruct-2 with thinking channel and offline-packed sequences.
deepseek_v4_flash_peft_openmath_thinking_packed_config	DSv4 Flash LoRA on packed OpenMathInstruct-2 thinking data.

Data

__all__
_DEEPSEEK_V4_LORA_TARGET_MODULES

API

bridge.recipes.deepseek.deepseek_v4.__all__

[‘deepseek_v4_flash_no_mtp_sft_config’, ‘deepseek_v4_flash_peft_openmath_thinking_packed_config’, ‘d…

bridge.recipes.deepseek.deepseek_v4._DEEPSEEK_V4_LORA_TARGET_MODULES

[‘linear_q_down_proj’, ‘linear_q_up_proj’, ‘linear_kv_proj’, ‘linear_proj’, ‘linear_fc1’, ‘linear_fc…

bridge.recipes.deepseek.deepseek_v4._apply_deepseek_v4_lora(

cfg: megatron.bridge.training.config.ConfigContainer,

) → None

Apply the DeepSeek V4 LoRA convergence contract to an SFT config.

bridge.recipes.deepseek.deepseek_v4.deepseek_v4_flash_sft_openmath_thinking_packed_config() → megatron.bridge.training.config.ConfigContainer

DSv4 Flash SFT on OpenMathInstruct-2 with thinking channel and offline-packed sequences.

CoT reasoning goes into the assistant thinking field and the final answer into the content field. Uses packed sequences for efficient training. Pre-pack data with prepare_gpt_sft_packed_data.py before running SFT. When using CP>1, pass model.cp_partition_mode=contiguous (required for DSv4 CSA attention) and pad_seq_to_mult=4 to ensure divisibility by cp_size.

For GB200-optimized training with HybridEP dispatcher and DSA kernel fusion, use deepseek_v4_flash_sft_openmath_thinking_packed_gb200_config instead.

bridge.recipes.deepseek.deepseek_v4.deepseek_v4_flash_peft_openmath_thinking_packed_config() → megatron.bridge.training.config.ConfigContainer

DSv4 Flash LoRA on packed OpenMathInstruct-2 thinking data.

The attention targets follow the DeepSeek MLA projection layout. Both shared and routed MLP projections are adapted; grouped routed experts use one adapter per local expert to match the verl training layout.