> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# Laguna

[Laguna](https://huggingface.co/poolside/Laguna-S-2.1) is Poolside's hybrid-attention MoE language model family. The Automodel implementation targets full SFT with expert parallelism for large Laguna checkpoints such as Laguna S 2.1.

|                  |                                             |
| ---------------- | ------------------------------------------- |
| **Task**         | Text Generation (MoE, hybrid attention)     |
| **Architecture** | `LagunaForCausalLM`                         |
| **Parameters**   | 118B total, \~8B activated for Laguna S 2.1 |
| **HF Org**       | [poolside](https://huggingface.co/poolside) |

## Architecture

* `LagunaForCausalLM`
* Layer-specific attention head counts, QK RMSNorm, and softplus attention output gating.
* Full and sliding-window attention layers can use separate RoPE settings.
* MoE blocks use `nemo_automodel.components.moe.layers.MoE` with sigmoid routing, top-k probability normalization, fp32 gate compute, correction bias loading, one shared expert, and grouped expert weights for EP.

## Example Recipes

| Recipe                                                                                                                                                   | Description                               |
| -------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |
| [laguna\_s\_2p1\_hellaswag\_ep16.yaml](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/laguna/laguna_s_2p1_hellaswag_ep16.yaml) | SFT — Laguna S 2.1 on HellaSwag with EP16 |

## Run the Recipe

```bash
uv run automodel --nproc-per-node=8 examples/llm_finetune/laguna/laguna_s_2p1_hellaswag_ep16.yaml
```

For multi-node launches, submit through your cluster launcher with two 8-GPU nodes for the default `ep_size: 16`.