NeMo Gym Documentation
NeMo Gym is a library for building reinforcement learning (RL) training environments for large language models (LLMs). NeMo Gym provides infrastructure to develop environments, scale rollout collection, and integrate seamlessly with your preferred training framework.
Install, run servers, and collect your first rollouts.
start here5 minBuild single-step, multi-step, and real-world environments.
Train with NeMo RL, OpenRLHF, Unsloth, and more.
Introduction to NeMo Gym
Understand NeMo Gym’s purpose and core components before diving into tutorials.
Motivation and benefits of NeMo Gym.
motivationbenefitsTraining approaches, core components, configuration, verification, and RL terminology.
sftrlrlvrenvironmentsagentsmodelsresourcesUnderstand how NeMo Gym fits within the RL environment ecosystem.
ecosystemintegrationsGet Started
Install and run NeMo Gym to start collecting rollouts.
Install, start servers, and collect your first rollouts in one page.
start here5 minStep-by-step installation with requirements, configuration, and troubleshooting.
15 minenvironmentconfigurationGenerate batches of scored interactions and view them with the rollout viewer.
10 minrolloutstraining-dataEnvironment Configuration
Configure and customize environment components and prepare datasets.
Orchestrate rollouts, tool calling, and verification.
orchestrationrolloutsConfigure LLM inference backends including vLLM.
inferencevllmDefine tasks, tools, and verification logic for your environment.
environmentsverificationPrepare and validate training datasets.
datasetsjsonlEnvironment Tutorials
Learn how to build custom training environments for various RL scenarios.
View all environment tutorials →Training Tutorials
Train models using NeMo Gym with your preferred RL framework.
Hands-on tutorials with NeMo RL, Unsloth, and more.
grpoRun multiple training environments simultaneously for rollout collection.
multi-environmentmulti-verifierTransform rollouts into SFT and DPO format.
sftdpoInfrastructure
Deploy NeMo Gym and plan cluster resources for training.
Contribute
Contribute to NeMo Gym development.
Contribute new environments or integrate existing benchmarks.
environmentsImplement NeMo Gym integration into a new training framework.
training-integration
