> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# MiMo-V2-Flash

> Fine-tune MiMo-V2-Flash and explore MiMo-V2.6-Flash-RL language and vision-language recipes, checkpoints, and settings with NeMo AutoModel.

[MiMo-V2-Flash](https://huggingface.co/XiaomiMiMo/MiMo-V2-Flash) is a Mixture-of-Experts (MoE) language model with a hybrid attention architecture. It interleaves sliding-window and global attention.

[MiMo-V2.6-Flash-RL](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL) uses the registered `MiMoV2ForCausalLM` model class. When `vision_config` is not `None`, the implementation sets `self.visual` to a `MiMoVisionTransformer`.

Set up NeMo AutoModel with the [latest container](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nemo-automodel) or follow the [installation instructions](/get-started/installation).

## Fine-Tune MiMo-V2-Flash

Use the checked-in recipes to fine-tune either supported checkpoint.

## Choose a Workflow

Choose a recipe for your checkpoint and workload. The MiMo-V2-Flash recipe is configured for 16 nodes with 8 H100 GPUs per node. The MiMo-V2.6-Flash-RL recipes target 8 nodes with 8 H100 GPUs per node.

| Goal                                              | Start Here                                                                                                                                                                                                                |
| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Fine-tune MiMo-V2-Flash on HellaSwag              | Use [mimo\_v2\_flash\_hellaswag.yaml](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/mimo_v2_flash/mimo_v2_flash_hellaswag.yaml).                                                               |
| Fine-tune MiMo-V2.6-Flash-RL on packed Tulu3 text | Use [mimo\_v2\_6\_flash\_rl\_tulu3\_packed4k\_ep64\_cp2\_100steps.yaml](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/mimo_v2_flash/mimo_v2_6_flash_rl_tulu3_packed4k_ep64_cp2_100steps.yaml). |
| Fine-tune MiMo-V2.6-Flash-RL on MedPix-VQA        | Use [mimo\_v2\_6\_flash\_rl\_medpix\_nonpacked4k\_ep64\_100steps.yaml](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/mimo_v2_flash/mimo_v2_6_flash_rl_medpix_nonpacked4k_ep64_100steps.yaml).  |

Follow the [launcher guide](/job-launchers/overview) to run a multi-node recipe.

## Model Reference

### Model Architecture

| Property                 | Value                                                                                                                       |
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------- |
| Model Implementation     | `MiMoV2FlashForCausalLM` backs MiMo-V2-Flash.                                                                               |
| Model Implementation     | `MiMoV2ForCausalLM` backs MiMo-V2.6-Flash-RL.                                                                               |
| Sliding-Window Attention | Sliding-window attention uses the `MiMoV2FlashAttention(is_swa=True)` path.                                                 |
| Expert Routing           | Expert routing maps `scoring_func="sigmoid"` to `score_func="sigmoid_with_bias"`; the recipes set `gate_precision=float32`. |

### Available Models

| Model              | Hugging Face ID                                                                         |
| ------------------ | --------------------------------------------------------------------------------------- |
| MiMo-V2-Flash      | [`XiaomiMiMo/MiMo-V2-Flash`](https://huggingface.co/XiaomiMiMo/MiMo-V2-Flash)           |
| MiMo-V2.6-Flash-RL | [`XiaomiMiMo/MiMo-V2.6-Flash-RL`](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL) |

## Related Resources

* [LLM Fine-Tuning Guide](/recipes-e2e-examples/sft-peft)
* [Launcher Guide](/job-launchers/overview)