> This page is for version 0.7.0 · 26.10. > For other versions, use one of these documentation indexes: > - Nightly (default): https://docs.nvidia.com/nemo/automodel/nightly/llms.txt > - Latest: https://docs.nvidia.com/nemo/automodel/latest/llms.txt > - 0.7.0 · 26.10: https://docs.nvidia.com/nemo/automodel/v0.7/llms.txt > - 0.5.0 · 26.06: https://docs.nvidia.com/nemo/automodel/v0.5/llms.txt > - 0.4.0 · 26.04: https://docs.nvidia.com/nemo/automodel/v0.4/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server. # Training Recipes > Choose a reusable NeMo AutoModel training workflow and find example configurations for fine-tuning, pretraining, distillation, retrieval, and diffusion models. Training recipes explain how to configure a workflow and adapt it to your model and data. For a walkthrough of a specific model or task, start with [Tutorials](/nemo/automodel/v0.7/tutorials/overview). To check model support and available configurations, see [Model Coverage](/nemo/automodel/v0.7/model-coverage/overview). NeMo AutoModel is organized around two key concepts: recipes and components. Recipes are executable scripts configured with YAML files. Each recipe defines its own training and validation loop, orchestrated through a `step_scheduler`. It specifies the model, dataset, loss function, optimizer, scheduler, checkpointing, and distributed training settings—allowing end-to-end training with a single command. Components are modular, plug-and-play building blocks referenced using the `_target_` field. These include models, datasets, loss functions, and distribution managers. Recipes assemble these components, making it easy to swap them out to change precision, distribution strategy, dataset, or task—without modifying the training loop itself. This page maps the ready-to-run recipes found in the `examples/` directory to their intended use cases, representative model families, and the most relevant how-to guides. * Examples root: [examples/ (GitHub)](https://github.com/NVIDIA-NeMo/Automodel/tree/r0.7.0/examples) * Getting started: [Install NeMo AutoModel](/nemo/automodel/v0.7/get-started/installation) ## Choose a Training Workflow | Workflow | Guide | | ---------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Supervised and parameter-efficient fine-tuning | [SFT & PEFT](/nemo/automodel/v0.7/recipes-e2e-examples/sft-peft) | | Pretraining | [Pretraining](/nemo/automodel/v0.7/recipes-e2e-examples/pretraining) | | Teacher-student training | [Knowledge Distillation](/nemo/automodel/v0.7/recipes-e2e-examples/knowledge-distillation) | | Classification | [Sequence Classification](/nemo/automodel/v0.7/recipes-e2e-examples/sequence-classification) | | Embedding and reranker training | [Retrieval Fine-Tuning](/nemo/automodel/v0.7/recipes-e2e-examples/retrieval-finetune) | | Large mixture-of-experts models | [Large MoE Fine-Tuning](/nemo/automodel/v0.7/recipes-e2e-examples/large-moe-fine-tuning) | | Image and video diffusion models | [Diffusion Fine-Tuning](/nemo/automodel/v0.7/recipes-e2e-examples/diffusion-fine-tuning) | | Diffusion language models | [dLLM Fine-Tuning](/nemo/automodel/v0.7/recipes-e2e-examples/dllm-fine-tuning) | | Quantization-aware training | [QAT](/nemo/automodel/v0.7/recipes-e2e-examples/qat) | | Speculative drafter training | [EAGLE](/nemo/automodel/v0.7/recipes-e2e-examples/eagle-speculative-decoding), [DFlash](/nemo/automodel/v0.7/recipes-e2e-examples/dflash-speculative-decoding), [DSpark](/nemo/automodel/v0.7/recipes-e2e-examples/dspark-speculative-decoding) | The sections below link to example configurations by model type. For help choosing an execution environment, see [Run Jobs](/nemo/automodel/v0.7/job-launchers/overview). ## Large Language Models (LLM) This section provides practical recipes and configurations for working with large language models across three core workflows: fine-tuning, pretraining, and knowledge distillation. ### Fine-Tuning End-to-end fine-tuning recipes for many open models. Each subfolder contains YAML configurations showing task setups (e.g., SQuAD, HellaSwag), precision options (e.g., FP8), and parameter-efficient methods (e.g., LoRA/QLoRA). * Folder: [examples/llm_finetune](https://github.com/NVIDIA-NeMo/Automodel/tree/r0.7.0/examples/llm_finetune) * Representative families: Llama 3.1/3.2/3.3, Gemma 2/3, Falcon 3, Mistral/Mixtral, Nemotron, Granite, Starcoder, Qwen, Baichuan, GLM, OLMo, Phi, GPT-OSS, Moonlight * How-to guide: [LLM finetuning](/nemo/automodel/v0.7/recipes-e2e-examples/sft-peft) ### Pretraining Starter configurations and scripts for pretraining with datasets from different stacks (e.g., PyTorch, Megatron Core). * Folder: [examples/llm_pretrain](https://github.com/NVIDIA-NeMo/Automodel/tree/r0.7.0/examples/llm_pretrain) * Example models: GPT-2 baseline, NanoGPT, DeepSeek-V3, Moonlight 16B TE (Slurm) * How-to guides: * [LLM pretraining](/nemo/automodel/v0.7/recipes-e2e-examples/pretraining) * [Pretraining with NanoGPT](/nemo/automodel/v0.7/recipes-e2e-examples/nanogpt-pretraining) ### Knowledge Distillation (KD) Recipes for distilling knowledge from a large teacher model into a smaller, more efficient student model. * Folder: [examples/llm_kd](https://github.com/NVIDIA-NeMo/Automodel/tree/r0.7.0/examples/llm_kd) * Example model: Llama 3.2 1B * How-to guide: [Knowledge distillation](/nemo/automodel/v0.7/recipes-e2e-examples/knowledge-distillation) ### Benchmark Configurations Curated configurations for benchmarking different training stacks and settings (e.g., Torch vs. TransformerEngine + DeepEP, various model sizes, MoE variants). * Folder: [examples/llm_benchmark](https://github.com/NVIDIA-NeMo/Automodel/tree/r0.7.0/examples/llm_benchmark) * Representative configurations: DeepSeek-V3, GPT-OSS (20B/120B), Kimi K2, Moonlight 16B, Qwen3 MoE 30B ## Vision Language Models (VLM) This section provides practical recipes and configurations for working with vision language models, covering fine-tuning and generation workflows for multimodal tasks. ### Fine-Tuning Fine-tuning recipes for VLMs. * Folder: [examples/vlm_finetune](https://github.com/NVIDIA-NeMo/Automodel/tree/r0.7.0/examples/vlm_finetune) * Representative family: Gemma 3 (various configurations) * Tutorial: [Fine-Tune Gemma 3 and Gemma 3n](/nemo/automodel/v0.7/recipes-e2e-examples/gemma-3-3n) ### Generation Simple generation script and configurations for VLMs. * Folder: [examples/vlm_generate](https://github.com/NVIDIA-NeMo/Automodel/tree/r0.7.0/examples/vlm_generate) ## Audio Models (ASR) This section provides recipes for fine-tuning omni / audio-capable models on automatic speech recognition (ASR) tasks. The recipes reuse the VLM training stack but operate on `{audio, text}` HuggingFace datasets (AMI, LibriSpeech, GigaSpeech, CommonVoice, etc.). ### Fine-Tuning End-to-end ASR fine-tuning of `Qwen3-Omni-30B-A3B-Instruct` on any HuggingFace audio dataset, including a thinker-only checkpoint export step for downstream `transformers` / vLLM loading. * Folder: [examples/audio_finetune/qwen3_omni_asr](https://github.com/NVIDIA-NeMo/Automodel/tree/r0.7.0/examples/audio_finetune/qwen3_omni_asr) * Representative model: Qwen3-Omni-30B-A3B-Instruct * Tutorial: [Fine-Tune Qwen3-Omni for ASR](/nemo/automodel/v0.7/recipes-e2e-examples/qwen3-omni-asr) ## Diffusion Models (Text-to-Image & Text-to-Video) Text-to-image and text-to-video diffusion models can generate visual content from natural language descriptions. Fine-tuning lets you adapt these models to a specific style, domain, or dataset — for example, generating product videos in your brand's aesthetic. Pretraining gives you full control when no existing model fits your needs. This section walks through the full workflow in NeMo AutoModel: preparing your dataset, training the model, and generating outputs. ### Fine-Tuning Fine-tuning recipes for adapting pretrained diffusion models to your data. * Folder: [examples/diffusion/finetune](https://github.com/NVIDIA-NeMo/Automodel/tree/r0.7.0/examples/diffusion/finetune) * Representative models: FLUX.1-dev (T2I, 12B), Wan 2.1 T2V 1.3B, HunyuanVideo 1.5 * How-to guide: [Diffusion fine-tuning](/nemo/automodel/v0.7/recipes-e2e-examples/diffusion-fine-tuning) ### Pretraining Pretraining recipes for training diffusion models from scratch on large-scale datasets. * Folder: [examples/diffusion/pretrain](https://github.com/NVIDIA-NeMo/Automodel/tree/r0.7.0/examples/diffusion/pretrain) * Representative models: Wan 2.1 T2V 1.3B, FLUX.1-dev * How-to guide: [Diffusion fine-tuning (pretraining section)](/nemo/automodel/v0.7/recipes-e2e-examples/diffusion-fine-tuning#configure-your-training-recipe) ### Generation Generation scripts and configs for running inference with pretrained or fine-tuned diffusion models. * Folder: [examples/diffusion/generate](https://github.com/NVIDIA-NeMo/Automodel/tree/r0.7.0/examples/diffusion/generate) * Representative models: Wan 2.1 1.3B, FLUX.1-dev, HunyuanVideo * How-to guide: [Diffusion generation](/nemo/automodel/v0.7/recipes-e2e-examples/diffusion-fine-tuning#generation--inference) ### Dataset Preparation Preprocessing pipeline to create `.meta` files containing VAE latents and text embeddings. * How-to guide: [Diffusion dataset preparation](/nemo/automodel/v0.7/datasets/diffusion-dataset) --- If you are new to the project, begin with the [Install NeMo AutoModel](/nemo/automodel/v0.7/get-started/installation) guide. Then, select a recipe category above and follow its linked how-to guide(s). The provided YAML configurations can serve as templates—customize them by adapting model names, datasets, and precision settings to match your specific needs.