Model CoverageLarge Language Models (LLMs)

Gemma

View as Markdown

Google’s Gemma is a family of open-weight language models built on the same research and technology as Gemini. Gemma models are available in multiple sizes and versions, with improvements in each generation including local sliding window attention (Gemma 2) and interleaved global/local attention (Gemma 3).

TaskText Generation
ArchitectureGemmaForCausalLM / Gemma2ForCausalLM / Gemma3ForCausalLM
Parameters1B – 27B
HF Orggoogle

Available Models

  • Gemma 3: 1B, 4B, 12B, 27B
  • Gemma 2: 2B, 9B, 27B
  • Gemma (v1): 2B, 7B

Architectures

  • GemmaForCausalLM — Gemma v1
  • Gemma2ForCausalLM — Gemma 2
  • Gemma3ForCausalLM — Gemma 3

Example HF Models

ModelHF ID
Gemma 1.1 2B ITgoogle/gemma-1.1-2b-it
Gemma 2Bgoogle/gemma-2b
Gemma 2 9B ITgoogle/gemma-2-9b-it
Gemma 2 27Bgoogle/gemma-2-27b
Gemma 3 1B ITgoogle/gemma-3-1b-it
Gemma 3 4B ITgoogle/gemma-3-4b-it
Gemma 3 27B ITgoogle/gemma-3-27b-it

Example Recipes

RecipeDescription
gemma_2_9b_it_squad.yamlSFT — Gemma 2 9B IT on SQuAD
gemma_2_9b_it_squad_peft.yamlLoRA — Gemma 2 9B IT on SQuAD
gemma_3_270m_squad.yamlSFT — Gemma 3 270M on SQuAD
gemma_3_270m_squad_peft.yamlLoRA — Gemma 3 270M on SQuAD

Try with NeMo AutoModel

1. Install (full instructions):

$pip install nemo-automodel

2. Clone the repo to get the example recipes:

$git clone https://github.com/NVIDIA-NeMo/Automodel.git
$cd Automodel

3. Run the recipe from inside the repo:

$automodel --nproc-per-node=8 examples/llm_finetune/gemma/gemma_2_9b_it_squad.yaml

1. Pull the container and mount a checkpoint directory:

$docker run --gpus all -it --rm \
>  --shm-size=8g \
>  -v $(pwd)/checkpoints:/opt/Automodel/checkpoints \
>  nvcr.io/nvidia/nemo-automodel:26.04.00

2. Navigate to the AutoModel directory (where the recipes are):

$cd /opt/Automodel

3. Run the recipe:

$automodel --nproc-per-node=8 examples/llm_finetune/gemma/gemma_2_9b_it_squad.yaml

See the Installation Guide and LLM Fine-Tuning Guide.

Fine-Tuning

See the LLM Fine-Tuning Guide for full SFT and LoRA instructions.

Hugging Face Model Cards