Model CoverageLarge Language ModelsMistral AIMistral-7B-v0.1

Mistral-7B-v0.1

View as Markdown

Mistral AI models are efficient transformer decoder models featuring sliding window attention for long context support. Mistral-Nemo is a 12B model developed jointly with NVIDIA.

Set up NeMo AutoModel with the latest container or follow the installation instructions.

Fine-Tune Mistral-7B-v0.1

From the repository root, run:

uv run automodel --nproc-per-node=8 examples/llm_finetune/mistral/mistral_7b_squad.yaml

Choose a Workflow

GoalStart Here
Supervised fine-tuning (SFT) - Mistral 7B on SQuADUse mistral_7b_squad.yaml.
Low-rank adaptation (LoRA) - Mistral 7B on SQuADUse mistral_7b_squad_peft.yaml.

Model Reference

Model Architecture

PropertyValue
TaskText Generation
ArchitectureMistralForCausalLM
Parameters7B
Hugging Face Organizationmistralai

Available Models

ModelHF ID
Mistral 7B v0.1mistralai/Mistral-7B-v0.1
Mistral 7B Instruct v0.1mistralai/Mistral-7B-Instruct-v0.1