Algorithms

NeMo RL supports multiple training algorithms for post-training large language models.

Support Matrix

Algorithms	Single Node	Multi-node
GRPO	GRPO Single Node	GRPO Multi-node: GRPO Qwen2.5-32B, GRPO Multi-Turn
DAPO	DAPO Single Node	DAPO Multi-node
CISPO	CISPO Configuration	CISPO Async Lag-1 Recipe
On-policy Distillation	Distillation Single Node	Distillation Multi-node
Multi-Teacher On-Policy Distillation (MOPD)	—	MOPD Configuration
Supervised Fine-Tuning (SFT)	SFT Single Node	SFT Multi-node
DPO	DPO Single Node	DPO Multi-node
PPO	PPO Single Node	PPO Multi-node
RM	RM Single Node	RM Multi-node

On-policy distillation is also supported in the PyTorch DTensor path.