NeMo Gym Documentation#
NeMo Gym is a library for building reinforcement learning (RL) training environments for large language models (LLMs). NeMo Gym provides infrastructure to develop environments, scale rollout collection, and integrate seamlessly with your preferred training framework.
Introduction to NeMo Gym#
Understand NeMo Gym’s purpose and core components before diving into tutorials.
Motivation and benefits of NeMo Gym.
Training approaches, core components, configuration, verification, and RL terminology.
Understand how NeMo Gym fits within the RL environment ecosystem.
Get Started#
Install and run NeMo Gym to start collecting rollouts.
Install, start servers, and collect your first rollouts in one page.
Step-by-step installation with requirements, configuration, and troubleshooting.
Generate batches of scored interactions and view them with the rollout viewer.
Environment Configuration#
Configure and customize environment components and prepare datasets.
Orchestrate rollouts, tool calling, and verification.
Configure LLM inference backends including vLLM.
Define tasks, tools, and verification logic for your environment.
Prepare and validate training datasets.
Environment Tutorials#
Learn how to build custom training environments for various RL scenarios.
Build a complete training environment from scratch.
Training Tutorials#
Train models using NeMo Gym with your preferred RL framework.
Hands-on tutorials with NeMo RL, Unsloth, and more.
Run multiple training environments simultaneously for rollout collection.
Transform rollouts into SFT and DPO format.
Infrastructure#
Deploy NeMo Gym and plan cluster resources for training.
Production deployment patterns and configurations.
Contribute#
Contribute to NeMo Gym development.
Contribute new environments or integrate existing benchmarks.
Implement NeMo Gym integration into a new training framework.