Model Training with Nemotron Steps#

This section documents how to run supervised fine-tuning (SFT), parameter-efficient fine-tuning (PEFT), reinforcement learning (RL) alignment, and post-training optimization with Nemotron steps. Each step packages a training approach, configuration files, and entry logic that you invoke through the nemotron steps CLI. For the definitions of step, configuration, and environment profile that apply across every domain, see Nemotron Steps Basics. If you are new to fine-tuning, start with Training Basics, which defines the training-specific terms the rest of this section uses.

Capabilities at a Glance#

Area

Step Names

Role

SFT

sft/automodel, sft/megatron_bridge

Supervised fine tuning from chat-formatted JSON Lines (JSONL) or packed Apache Parquet

PEFT

peft/automodel, peft/megatron_bridge

Adapter training with a smaller trainable surface

RL

rl/nemo_rl/dpo, rl/nemo_rl/rlvr, rl/nemo_rl/rlhf

Alignment after a supervised fine tuning (SFT) policy exists

Optimize

optimize/modelopt/quantize, optimize/modelopt/prune, optimize/modelopt/distill

Compression and quality recovery

Limitations and Restrictions#

The Nemotron steps for data preparation and model training do not support local training, such as on a developer workstation.

These steps require access to at least two nodes, each equipped with 8 x NVIDIA A100 80 GB or better GPUs. These steps support the following environments:

  • Slurm

  • NVIDIA DGX Cloud Lepton

  • NVIDIA Run:ai

For assistance with configuring access to one of the supported computing environments, refer to Env Profile Generator or run the nemotron-env-toml skill with your agent.

Learning Path#

Training Basics

This page defines supervised fine-tuning, parameter-efficient fine-tuning, reinforcement learning alignment, quantization, tokenizers, the chat dataset format, and checkpoints.

Beginner Level

Training Basics
Getting Started

This guide covers installation expectations, the environment profile, and how to run your first sample job with the tiny configuration.

Beginner

Getting Started with Training Steps
How-To Guides

These guides are task-focused. They explain how to pick a backend, wire data, and run optimization steps.

Intermediate

Model Training How-To Guides
Explanation

This material explains the basics, the artifact graph, and how the training libraries differ.

Concepts

Training Concepts
Reference

This material is for lookup. It lists the step catalog, parameters, and configuration conventions.

Lookup

Training Reference
Model Training with Agents

Use the customize skill with a YAML-first plan: repo steps, then configs, then code only for gaps.

Workflow

Tips for Model Training with Agents