Model CoverageLarge Language Models

Gemma

View as Markdown

Google’s Gemma is a family of open-weight language models built on the same research and technology as Gemini. Gemma models are available in multiple sizes and versions, with improvements in each generation including local sliding window attention (Gemma 2), interleaved global/local attention (Gemma 3), and the unified Gemma 4 architecture.

TaskText Generation
ArchitectureGemmaForCausalLM / Gemma2ForCausalLM / Gemma3ForCausalLM / Gemma4UnifiedForConditionalGeneration
Parameters1B – 27B
HF Orggoogle

Available Models

  • Gemma 4: 12B
  • Gemma 3: 1B, 4B, 12B, 27B
  • Gemma 2: 2B, 9B, 27B
  • Gemma (v1): 2B, 7B

Architectures

  • GemmaForCausalLM — Gemma v1
  • Gemma2ForCausalLM — Gemma 2
  • Gemma3ForCausalLM — Gemma 3
  • Gemma4UnifiedForConditionalGeneration — Gemma 4 dense

Example HF Models

ModelHF ID
Gemma 1.1 2B ITgoogle/gemma-1.1-2b-it
Gemma 2Bgoogle/gemma-2b
Gemma 2 9B ITgoogle/gemma-2-9b-it
Gemma 2 27Bgoogle/gemma-2-27b
Gemma 3 1B ITgoogle/gemma-3-1b-it
Gemma 3 4B ITgoogle/gemma-3-4b-it
Gemma 3 27B ITgoogle/gemma-3-27b-it
Gemma 4 12Bgoogle/gemma-4-12B

Example Recipes

RecipeDescription
gemma_2_9b_it_squad.yamlSFT — Gemma 2 9B IT on SQuAD
gemma_2_9b_it_squad_peft.yamlLoRA — Gemma 2 9B IT on SQuAD
gemma_3_270m_squad.yamlSFT — Gemma 3 270M on SQuAD
gemma_3_270m_squad_peft.yamlLoRA — Gemma 3 270M on SQuAD
gemma_4_12b_hellaswag.yamlSFT — Gemma 4 12B on HellaSwag

Try with NeMo AutoModel

1. Install (full instructions):

$pip install nemo-automodel

2. Clone the repo to get the example recipes:

$git clone https://github.com/NVIDIA-NeMo/Automodel.git
$cd Automodel

3. Run the recipe from inside the repo:

$automodel --nproc-per-node=8 examples/llm_finetune/gemma/gemma_2_9b_it_squad.yaml

1. Pull the container and mount a checkpoint directory:

$docker run --gpus all -it --rm \
>  --shm-size=8g \
>  -v $(pwd)/checkpoints:/opt/Automodel/checkpoints \
>  nvcr.io/nvidia/nemo-automodel:26.06.00

2. Navigate to the AutoModel directory (where the recipes are):

$cd /opt/Automodel

3. Run the recipe:

$automodel --nproc-per-node=8 examples/llm_finetune/gemma/gemma_2_9b_it_squad.yaml

See the Installation Guide and LLM Fine-Tuning Guide.

Fine-Tuning

See the LLM Fine-Tuning Guide for full SFT and LoRA instructions.

Hugging Face Model Cards