Mixtral-8x7B-v0.1

Mixtral is Mistral AI’s Mixture-of-Experts model series. Each token is processed by a subset of experts, enabling a large total parameter count with efficient per-token compute.

Set up NeMo AutoModel with the latest container or follow the installation instructions.

Fine-Tune Mixtral-8x7B-v0.1

From the repository root, run:

uv run automodel --nproc-per-node=8 examples/llm_finetune/mistral/mixtral-8x7b-v0-1_squad.yaml

Choose a Workflow

Model Reference

Model Architecture

Available Models

Related Resources

• LLM Fine-Tuning Guide
• Large MoE Fine-Tuning Guide