Inkling
Inkling is a multimodal Mixture-of-Experts model family from Thinking Machines Lab. The models accept text, image, video, and audio inputs and generate text. NeMo AutoModel supports both the 975B Inkling checkpoint and the 276B Inkling-Small checkpoint.
Architecture
Inkling uses a 66-layer decoder with hybrid local and global attention. Inkling-Small uses the same model family with 42 decoder layers and hidden size 4096. Local attention uses a 512-token sliding window, and every sixth layer uses global attention. Both variants use eight key-value heads and short convolutions with kernel size four.
Each sparse feed-forward layer routes a token to 6 of 256 experts and also evaluates 2 shared experts. Images and video use a hierarchical patch encoder, while audio uses discrete dMel tokens.
NeMo AutoModel retains the checkpoint’s fused expert layout and supports expert parallelism for both variants. The full Inkling recipe also uses pipeline parallelism.
Example HF Models
Example Recipes
Both recipes require a multi-node launch. Adjust the data and parallel dimensions to the available cluster before running a recipe.
The Inkling-Small EP64 recipe completed 100 training steps on 64 H100 GPUs with microbatch size 1, global batch size 64, and 512-token MedPix samples. It uses FSDP2, activation checkpointing, HybridEP dispatch, and no context or pipeline parallelism.