Model CoverageLarge Language ModelsIBMgranite-3.0-1b-a400m-base

granite-3.0-1b-a400m-base

View as Markdown

IBM Granite MoE models extend the Granite architecture with Mixture-of-Experts layers for more efficient scaling. PowerMoE (IBM Research) also uses this architecture.

Use this page as a checkpoint and architecture reference. Set up NeMo AutoModel with the latest container or follow the installation instructions.

Model Reference

Model Architecture

PropertyValue
TaskText Generation (MoE)
ArchitectureGraniteMoeForCausalLM
Parameters1B total / 400M active
Hugging Face Organizationibm-granite
  • GraniteMoeForCausalLM
  • GraniteMoeSharedForCausalLM - variant with shared experts

Available Models

ModelHF ID
Granite 3.0 1B A400M Baseibm-granite/granite-3.0-1b-a400m-base