Model CoverageOmniMicrosoftPhi-4-multimodal-instruct

Phi-4-multimodal-instruct

View as Markdown

Phi-4-multimodal-instruct is Microsoft’s multimodal extension of Phi-4, supporting text, image, and audio inputs - making it suitable for speech, vision, and combined multimodal tasks.

Set up NeMo AutoModel with the latest container or follow the installation instructions.

Fine-Tune Phi-4-multimodal-instruct

From the repository root, run:

uv run automodel --nproc-per-node=8 examples/vlm_finetune/phi4/phi4_mm_cv17.yaml

Choose a Workflow

GoalStart Here
Supervised fine-tuning (SFT) - Phi-4-multimodal on CommonVoice (audio-text)Use phi4_mm_cv17.yaml. Dataset: CommonVoice 17.

Model Reference

Model Architecture

PropertyValue
TaskOmnimodal (Text/Image/Audio)
ArchitecturePhi4MultimodalForCausalLM
Parameters5.6B
Hugging Face Organizationmicrosoft

Available Models

ModelHF ID
Phi-4-multimodal-instructmicrosoft/Phi-4-multimodal-instruct