Skip to main content

Ctrl+K

NeMo-RL

GitHub

NeMo-RL

GitHub

Table of Contents

About

Overview
Performance
Model Support
Features and Roadmap
Training and Generation Backends
Quick Start
Installation and Prerequisites
Algorithms
Evaluation
Set Up Clusters
Tips and Tricks

Environment Start

Run on Your Local Workstation
Set Up Clusters

E2E Examples

SFT on OpenMathInstruct-2

Guides

Nemotron 3 Nano
Nemotron 3 Nano Omni
Nemotron 3 Super
Add New Models
Supervised Fine-Tuning in NeMo RL
Direct Preference Optimization in NeMo RL
An in-depth Walkthrough of DAPO in NeMo RL
LoRA (Low-Rank Adaptation)
An in-depth Walkthrough of CISPO in NeMo RL
An In-Depth Walkthrough of ProRLv2 in NeMo RL
Two-Stage SWE RL for Qwen3-30B-A3B-Thinking
An In-depth Walkthrough of GRPO in NeMo RL
An In-Depth Walkthrough of PPO in NeMo RL
GRPO on DeepScaler
Solve a Sliding Puzzle Using GRPO
Audio Post-training with NeMo RL
Reward Model Training in NeMo RL
Environments for GRPO Training
Evaluation
DeepSeek-V3
Model Quirks
Train with Async GRPO
Quantization-Aware RL (QARL)
Train with Eagle3 Speculative Decoding
YaRN Long-Context Training
Cross-Tokenizer (X-Token) Off-Policy Distillation
Router Replay
Muon Optimizer
DTensor Tensor Parallel Accuracy Issue
Fault Tolerance Launcher Guide

Containers

Build Docker Images

Development

CI/CD
Test NeMo RL
Documentation Development
Debug NeMo RL Applications
Profile GPU with Nsys
FP8 Quantization in NeMo RL
Experiment with Custom vLLM

Design Docs

Design and Philosophy
Padding in NeMo RL
Logger
uv in NeMo RL
Dependency Management
Data Format
Generation Interface
Exporting Checkpoints to Hugging Face Format
Loss functions in NeMo RL
FSDP2 Parallel Plan
Training Backends
Sequence Packing and Dynamic Batching
Environment Variable Precedence in NeMo RL
NeMo Gym Integration

API Reference

API Reference
- nemo_rl

API Reference
nemo_rl
nemo_rl.data_plane
nemo_rl.data_plane.adapters

`nemo_rl.data_plane.adapters`#

Submodules#

nemo_rl.data_plane.adapters.transfer_queue
nemo_rl.data_plane.adapters.noop

previous

nemo_rl.data_plane

next

nemo_rl.data_plane.adapters.transfer_queue

On this page

Submodules

Privacy Policy | Your Privacy Choices | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2026, NVIDIA Corporation.