Ling 2.0#
Ling 2.0, documented in this repository under the Bailing family, is a high-sparsity Mixture-of-Experts language model family from inclusionAI. Megatron Bridge supports the Ling 2.0 MoE checkpoints through the Bailing bridge with auto-detected Hugging Face configuration and weight mapping.
Supported Variants#
Variant |
Hugging Face ID |
Notes |
|---|---|---|
Ling-flash-2.0 |
|
100B total, 6.1B active |
Ling-flash-base-2.0 |
|
Base checkpoint, 100B total, 6.1B active |
Ling-mini-2.0 |
|
16B total, 1.5B active |
Ling-mini-base-2.0 |
|
Base checkpoint, 16B total, 1.5B active |
Architecture Notes#
High-sparsity MoE with 256 routed experts and top-8 routing.
Sigmoid routing with QK-Norm and Half RoPE.
Custom Hugging Face model code is required, so conversion and inference commands use
--trust-remote-code.
Examples#
For checkpoint import/export, round-trip validation, and inference commands, see the Bailing examples README.