NeMo Gym Documentation

NeMo Gym is a library for building reinforcement learning (RL) training environments for large language models (LLMs). NeMo Gym provides infrastructure to develop environments, scale rollout collection, and integrate seamlessly with your preferred training framework.

Install, run servers, and collect your first rollouts.

start here5 min

Environment Tutorials

Build single-step, multi-step, and real-world environments.

Training Tutorials

Train with NeMo RL, OpenRLHF, Unsloth, and more.

Introduction to NeMo Gym

Understand NeMo Gym’s purpose and core components before diving into tutorials.

Motivation and benefits of NeMo Gym.

motivationbenefits

Training approaches, core components, configuration, verification, and RL terminology.

sftrlrlvrenvironmentsagentsmodelsresources

Understand how NeMo Gym fits within the RL environment ecosystem.

ecosystemintegrations

Get Started

Install and run NeMo Gym to start collecting rollouts.

Install, start servers, and collect your first rollouts in one page.

start here5 min

Detailed Setup Guide

Step-by-step installation with requirements, configuration, and troubleshooting.

15 minenvironmentconfiguration

Rollout Collection

Generate batches of scored interactions and view them with the rollout viewer.

10 minrolloutstraining-data

Environment Configuration

Configure and customize environment components and prepare datasets.

Orchestrate rollouts, tool calling, and verification.

orchestrationrollouts

Configure LLM inference backends including vLLM.

Resources Server

Define tasks, tools, and verification logic for your environment.

environmentsverification

Prepare and validate training datasets.

Environment Tutorials

Learn how to build custom training environments for various RL scenarios.

Building Environments

Build a complete training environment from scratch.

beginnerfoundational

View all environment tutorials →

Training Tutorials

Train models using NeMo Gym with your preferred RL framework.

Hands-on tutorials with NeMo RL, Unsloth, and more.

Multi-Environment Training

Run multiple training environments simultaneously for rollout collection.

multi-environmentmulti-verifier

Transform rollouts into SFT and DPO format.

View all training tutorials →

Infrastructure

Deploy NeMo Gym and plan cluster resources for training.

Deployment Topology

Production deployment patterns and configurations.

deploymenttopology

Contribute

Contribute to NeMo Gym development.

Contribute Environments

Contribute new environments or integrate existing benchmarks.

Integrate RL Frameworks

Implement NeMo Gym integration into a new training framework.

training-integration