Skip to main content

Ctrl+K

NeMo-RL

GitHub

NeMo-RL

GitHub

Table of Contents

About

Overview
Performance
Model Support
Features and Roadmap
Training and Generation Backends
Quick Start
Installation and Prerequisites
Algorithms
Evaluation
Set Up Clusters
Tips and Tricks

Environment Start

Run on Your Local Workstation
Set Up Clusters

E2E Examples

SFT on OpenMathInstruct-2

Guides

Add New Models
Supervised Fine-Tuning in NeMo RL
Direct Preference Optimization in NeMo RL
An in-depth Walkthrough of DAPO in NeMo RL
LoRA (Low-Rank Adaptation)
An in-depth Walkthrough of CISPO in NeMo RL
An In-Depth Walkthrough of ProRLv2 in NeMo RL
Two-Stage SWE RL for Qwen3-30B-A3B-Thinking
An In-depth Walkthrough of GRPO in NeMo RL
An In-Depth Walkthrough of PPO in NeMo RL
GRPO on DeepScaler
Solve a Sliding Puzzle Using GRPO
Audio Post-training with NeMo RL
Audio-Visual GRPO with Qwen2.5-Omni-7B
Reward Model Training in NeMo RL
Environments for GRPO Training
Evaluation
DeepSeek-V3
Model Guides
- Nemotron
- Qwen
  - Qwen3.5
Model Quirks
Train with Async GRPO
Train with Single-Controller (Async GRPO)
Quantization-Aware RL (QARL)
Train with Eagle3 Speculative Decoding
YaRN Long-Context Training
Cross-Tokenizer (X-Token) Off-Policy Distillation
Weight Refit: Choosing a Transport
Checkpoint-Engine Refit
Managed Dynamo generation on Slurm
Router Replay
Muon Optimizer
DTensor Tensor Parallel Accuracy Issue
Fault Tolerance Launcher Guide

Containers

Build Docker Images

Development

CI/CD
Test NeMo RL
Documentation Development
Debug NeMo RL Applications
Profile GPU with Nsys
FP8 Quantization in NeMo RL
Experiment with Custom vLLM

Design Docs

Design and Philosophy
Padding in NeMo RL
Logger
uv in NeMo RL
Dependency Management
Data Format
Generation Interface
Managed Dynamo generation design
Remote Sparse-Delta vLLM Refit
Checkpoint Engine Design
Exporting Checkpoints to Hugging Face Format
Loss functions in NeMo RL
FSDP2 Parallel Plan
Training Backends
Sequence Packing and Dynamic Batching
Environment Variable Precedence in NeMo RL
NeMo Gym Integration
ModelOpt Real-Quant Refit Architecture
NCCL Reshard Refit (Experimental)
Training on text that contains <image>

API Reference

API Reference
- nemo_rl

API Reference
nemo_rl
nemo_rl.utils
nemo_rl.utils.checkpoint_engines

`nemo_rl.utils.checkpoint_engines`#

Submodules#

nemo_rl.utils.checkpoint_engines.nixl
nemo_rl.utils.checkpoint_engines.base

previous

nemo_rl.utils

next

nemo_rl.utils.checkpoint_engines.nixl

On this page

Submodules

Privacy Policy | Your Privacy Choices | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2026, NVIDIA Corporation.