Model CoverageLarge Language ModelsAllen AIOLMoE-1B-7B-0924

OLMoE-1B-7B-0924

View as Markdown

OLMoE is Allen AI’s open Mixture-of-Experts language model. It activates 1B parameters per token from a 7B total parameter pool.

Use this page as a checkpoint and architecture reference. Set up NeMo AutoModel with the latest container or follow the installation instructions.

Model Reference

Model Architecture

PropertyValue
TaskText Generation (MoE)
ArchitectureOLMoEForCausalLM
Parameters7B total / 1B active
Hugging Face Organizationallenai

Available Models

ModelHF ID
OLMoE 1B 7Ballenai/OLMoE-1B-7B-0924
OLMoE 1B 7B Instructallenai/OLMoE-1B-7B-0924-Instruct