`bridge.models.kimi.kimi_k3_provider`#

Model provider for the Kimi K3 language backbone.

Module Contents#

Classes#

KimiK3ModelProvider

Megatron configuration and provider for Kimi K3.

API#

class bridge.models.kimi.kimi_k3_provider.KimiK3ModelProvider#

Bases: megatron.bridge.models.mla_provider.MLAModelProvider

Megatron configuration and provider for Kimi K3.

variable_seq_lengths: bool#: True

kimi_kda_layers: tuple[int, ...]#: ()

kimi_linear_num_heads: int#: 96

kimi_linear_head_dim: int#: 128

kimi_linear_conv_kernel_size: int#: 4

kimi_kda_gate_lower_bound: float#: None

kimi_attn_res_block_size: int#: 12