Mistral-Small-4-119B-2603 | NVIDIA NeMo AutoModel

Mistral-Small-4-119B is Mistral AI’s multimodal MoE model supporting both text and image inputs at scale.

Set up NeMo AutoModel with the latest container or follow the installation instructions.

Fine-Tune Mistral-Small-4-119B-2603

This recipe was validated on 4 nodes x 8 GPUs (32 H100s). See the Launcher Guide for multi-node setup.

From the repository root, run:

uv run automodel --nproc-per-node=8 examples/vlm_finetune/mistral4/mistral4_medpix.yaml

Goal	Start Here
Supervised fine-tuning (SFT) - Mistral-Small-4 on MedPix	Use mistral4_medpix.yaml. Dataset: MedPix-VQA.

Property	Value
Task	Image-Text-to-Text
Architecture	`MistralForConditionalGeneration`
Parameters	119B (MoE)
Hugging Face Organization	mistralai

Model	HF ID
Mistral-Small-4 119B	`mistralai/Mistral-Small-4-119B-2603`