Skip to main content

Ctrl+K

NeMo-RL

GitHub

NeMo-RL

GitHub

Table of Contents

About

Overview
Performance
Model Support
Features and Roadmap
Training and Generation Backends
Quick Start
Installation and Prerequisites
Algorithms
Evaluation
Set Up Clusters
Tips and Tricks

Environment Start

Run on Your Local Workstation
Set Up Clusters

E2E Examples

SFT on OpenMathInstruct-2

Guides

Add New Models
Supervised Fine-Tuning in NeMo RL
Direct Preference Optimization in NeMo RL
Nemotron Omni MPO
An in-depth Walkthrough of DAPO in NeMo RL
LoRA (Low-Rank Adaptation)
An in-depth Walkthrough of CISPO in NeMo RL
An In-Depth Walkthrough of ProRLv2 in NeMo RL
Two-Stage SWE RL for Qwen3-30B-A3B-Thinking
An In-depth Walkthrough of GRPO in NeMo RL
Length Penalties for GRPO (NeMo-Gym)
An In-Depth Walkthrough of PPO in NeMo RL
GRPO on DeepScaler
Solve a Sliding Puzzle Using GRPO
Audio Post-training with NeMo RL
Audio-Visual GRPO with Qwen2.5-Omni-7B
Reward Model Training in NeMo RL
Environments for GRPO Training
DeepSeek-V3
Model Guides
Model Quirks
Train with Async GRPO
Train with Single-Controller (Async GRPO and PPO)
Quantization-Aware RL (QARL)
Train with Eagle3 Speculative Decoding
Speculative Decoding Draft Co-Training on Automodel Backend
YaRN Long-Context Training
Cross-Tokenizer (X-Token) Off-Policy Distillation
Weight Refit: Choosing a Transport
Checkpoint-Engine Refit
Managed Dynamo generation on Slurm
Router Replay
Muon Optimizer
Automodel Tensor Parallel Accuracy Issue
Fault Tolerance Launcher Guide

Containers

Build Docker Images

Development

CI/CD
Test NeMo RL
Documentation Development
Debug NeMo RL Applications
Profile GPU with Nsys
FP8 Quantization in NeMo RL
Experiment with Custom vLLM

Observability

Observability

Design Docs

Design and Philosophy
Padding in NeMo RL
Logger
uv in NeMo RL
Dependency Management
Data Format
Generation Interface
Colocated PPO streaming
NemoRL<>TRTLLM Disaggregation Rollout (Experimental)
Managed Dynamo generation design
Remote Sparse-Delta vLLM Refit
Checkpoint Engine Design
Exporting Checkpoints to Hugging Face Format
Loss functions in NeMo RL
Automodel Parallel Plan
Training Backends
Sequence Packing and Dynamic Batching
Environment Variable Precedence in NeMo RL
NeMo Gym Integration
ModelOpt Real-Quant Refit Architecture
End-to-End W4A4 NVFP4 with Per-Token vLLM Rollout
NCCL Reshard Refit (Experimental)
DSpark Auxiliary Heads
Training on text that contains <image>
Automodel v0.6.0 and Context-Parallel Integration
Token Capture Lineage Ledger

API Reference

API Reference
- nemo_rl

API Reference
nemo_rl
nemo_rl.modelopt
nemo_rl.modelopt.models
nemo_rl.modelopt.models.policy

`nemo_rl.modelopt.models.policy`#

Subpackages#

nemo_rl.modelopt.models.policy.workers

previous

nemo_rl.modelopt.models.generation.vllm_quant_backend

next

nemo_rl.modelopt.models.policy.workers

On this page

Subpackages

Privacy Policy | Your Privacy Choices | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2026, NVIDIA Corporation.