Quick Start#

Use this quick start to get going with either the native PyTorch DTensor or Megatron Core training backends.

Note

Both training backends are independent — you can install and use either one on its own.

For more examples and setup details, continue to the Prerequisites section.

Quick Start Options#

Native PyTorch (DTensor)

Megatron Core

Clone and create the environment

 

git clone git@github.com:NVIDIA-NeMo/RL.git nemo-rl
cd nemo-rl
git submodule update --init --recursive
uv venv

Note

If you previously ran without checking out the submodules, you may need to rebuild virtual environments by setting NRL_FORCE_REBUILD_VENVS=true. See Tips and Tricks.

Native PyTorch (DTensor)

Megatron Core

Run GRPO (DTensor)

Run GRPO (Megatron)

 
# DTensor
uv run python examples/run_grpo_math.py

# Megatron
uv run examples/run_grpo_math.py \
  --config examples/configs/grpo_math_1B_megatron.yaml
On this page