Recipes and End-to-End Examples#

NeMo Automodel is organized around two key concepts: recipes and components.

Recipes are executable scripts configured with YAML files. Each recipe defines its own training and validation loop, orchestrated through a step_scheduler. It specifies the model, dataset, loss function, optimizer, scheduler, checkpointing, and distributed training settings—allowing end-to-end training with a single command.

Components are modular, plug-and-play building blocks referenced using the _target_ field. These include models, datasets, loss functions, and distribution managers. Recipes assemble these components, making it easy to swap them out to change precision, distribution strategy, dataset, or task—without modifying the training loop itself.

This page maps the ready-to-run recipes found in the examples/ directory to their intended use cases, representative model families, and the most relevant how-to guides.

Large Language Models (LLM)#

This section provides practical recipes and configurations for working with large language models across three core workflows: fine-tuning, pretraining, and knowledge distillation.

Fine-Tuning#

End-to-end fine-tuning recipes for many open models. Each subfolder contains YAML configurations showing task setups (e.g., SQuAD, HellaSwag), precision options (e.g., FP8), and parameter-efficient methods (e.g., LoRA/QLoRA).

  • Folder: examples/llm_finetune

  • Representative families: Llama 3.1/3.2/3.3, Gemma 2/3, Falcon 3, Mistral/Mixtral, Nemotron, Granite, Starcoder, Qwen, Baichuan, GLM, OLMo, Phi, GPT-OSS, Moonlight

  • How-to guide: LLM finetuning

Pretraining#

Starter configurations and scripts for pretraining with datasets from different stacks (e.g., PyTorch, Megatron Core).

Knowledge Distillation (KD)#

Recipes for distilling knowledge from a large teacher model into a smaller, more efficient student model.

Benchmark Configurations#

Curated configurations for benchmarking different training stacks and settings (e.g., Torch vs. TransformerEngine + DeepEP, various model sizes, MoE variants).

  • Folder: examples/benchmark/configs

  • Representative configurations: DeepSeek-V3, GPT-OSS (20B/120B), Kimi K2, Moonlight 16B, Qwen3 MoE 30B

Vision Language Models (VLM)#

This section provides practical recipes and configurations for working with vision language models, covering fine-tuning and generation workflows for multimodal tasks.

Fine-Tuning#

Fine-tuning recipes for VLMs.

Generation#

Simple generation script and configurations for VLMs.

Diffusion Models (Text-to-Image & Text-to-Video)#

Text-to-image and text-to-video diffusion models can generate visual content from natural language descriptions. Fine-tuning lets you adapt these models to a specific style, domain, or dataset — for example, generating product videos in your brand’s aesthetic. Pretraining gives you full control when no existing model fits your needs.

This section walks through the full workflow in NeMo AutoModel: preparing your dataset, training the model, and generating outputs.

Fine-Tuning#

Fine-tuning recipes for adapting pretrained diffusion models to your data.

Pretraining#

Pretraining recipes for training diffusion models from scratch on large-scale datasets.

Generation#

Generation scripts and configs for running inference with pretrained or fine-tuned diffusion models.

Dataset Preparation#

Preprocessing pipeline to create .meta files containing VAE latents and text embeddings.

If you are new to the project, begin with the Installation guide. Then, select a recipe category above and follow its linked how-to guide(s). The provided YAML configurations can serve as templates—customize them by adapting model names, datasets, and precision settings to match your specific needs.