Training Tutorials

We have hands-on tutorials with supported training frameworks to help you train with NeMo Gym environments. If you’re interested in integrating another training framework, see the Training Framework Integration Guide.

RL (GRPO)

NeMo RL

Tutorial-series: GRPO training to improve multi-step tool calling on the Workplace Assistant environment, scaling from single-node to multi-node training.

nemo rl grpo 3-5 hours

RL Training with NeMo RL using GRPO

OpenRLHF

Review the agent executor for using NeMo Gym environments with OpenRLHF.

openrlhf

https://github.com/OpenRLHF/OpenRLHF/blob/main/examples/python/agent_func_nemogym_executor.py

TRL

GRPO training on Workplace Assistant and Reasoning Gym environments

trl

RL Training with TRL

Unsloth

GRPO training on instruction following and reasoning environments.

unsloth single-gpu 30 min

RL Training with Unsloth

NeMo Customizer

Coming soon

nemo customizer in progress

VeRL

Coming soon

verl in progress

SFT & DPO

Offline Training with Rollouts

Transform rollouts into training data for supervised fine-tuning (SFT) and direct preference optimization (DPO).

sft dpo

Offline Training with Rollouts (SFT/DPO) - Experimental