Skip to main content

Ctrl+K

NeMo-RL

GitHub

NeMo-RL

GitHub

Table of Contents

About

Overview
Performance
Model Support
Features and Roadmap
Training and Generation Backends
Quick Start
Installation and Prerequisites
Algorithms
Evaluation
Set Up Clusters
Tips and Tricks

Environment Start

Run on Your Local Workstation
Set Up Clusters

E2E Examples

SFT on OpenMathInstruct-2

Guides

Nemotron 3 Nano
Nemotron 3 Nano Omni
Nemotron 3 Super
Nemotron 3 Ultra
Add New Models
Supervised Fine-Tuning in NeMo RL
Direct Preference Optimization in NeMo RL
An in-depth Walkthrough of DAPO in NeMo RL
LoRA (Low-Rank Adaptation)
An in-depth Walkthrough of CISPO in NeMo RL
An In-Depth Walkthrough of ProRLv2 in NeMo RL
Two-Stage SWE RL for Qwen3-30B-A3B-Thinking
An In-depth Walkthrough of GRPO in NeMo RL
An In-Depth Walkthrough of PPO in NeMo RL
GRPO on DeepScaler
Solve a Sliding Puzzle Using GRPO
Audio Post-training with NeMo RL
Audio-Visual GRPO with Qwen2.5-Omni-7B
Reward Model Training in NeMo RL
Environments for GRPO Training
Evaluation
DeepSeek-V3
Model Guides
Qwen
Qwen3.5
Model Quirks
Train with Async GRPO
Quantization-Aware RL (QARL)
Train with Eagle3 Speculative Decoding
YaRN Long-Context Training
Cross-Tokenizer (X-Token) Off-Policy Distillation
Weight Refit: Choosing a Transport
Checkpoint-Engine Refit
Router Replay
Muon Optimizer
DTensor Tensor Parallel Accuracy Issue
Fault Tolerance Launcher Guide

Containers

Build Docker Images

Development

CI/CD
Test NeMo RL
Documentation Development
Debug NeMo RL Applications
Profile GPU with Nsys
FP8 Quantization in NeMo RL
Experiment with Custom vLLM

Design Docs

Design and Philosophy
Padding in NeMo RL
Logger
uv in NeMo RL
Dependency Management
Data Format
Generation Interface
Remote Sparse-Delta vLLM Refit
Checkpoint Engine Design
Exporting Checkpoints to Hugging Face Format
Loss functions in NeMo RL
FSDP2 Parallel Plan
Training Backends
Sequence Packing and Dynamic Batching
Environment Variable Precedence in NeMo RL
NeMo Gym Integration
ModelOpt Real-Quant Refit Architecture
NCCL Reshard Refit (Experimental)

API Reference

API Reference
- nemo_rl

API Reference
nemo_rl
nemo_rl.models
nemo_rl.models.policy
nemo_rl.models.policy.workers

`nemo_rl.models.policy.workers`#

Submodules#

nemo_rl.models.policy.workers.megatron_remote_sparse_refit
nemo_rl.models.policy.workers.checkpoint_engine
nemo_rl.models.policy.workers.dtensor_policy_worker
nemo_rl.models.policy.workers.dtensor_policy_worker_v2
nemo_rl.models.policy.workers.megatron_policy_worker
nemo_rl.models.policy.workers.base_policy_worker
nemo_rl.models.policy.workers.patches

previous

nemo_rl.models.policy

next

nemo_rl.models.policy.workers.megatron_remote_sparse_refit

On this page

Submodules

Privacy Policy | Your Privacy Choices | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2026, NVIDIA Corporation.