NeMo Gym Documentation#
NeMo Gym is a library for building reinforcement learning (RL) training environments for large language models (LLMs). NeMo Gym provides infrastructure to develop environments, scale rollout collection, and integrate seamlessly with your preferred training framework.
A training environment consists of three server components: Agents orchestrate the rollout lifecycle—calling models, executing tool calls through resources, and coordinating verification. Models provide stateless text generation using LLM inference endpoints. Resources define tasks, tool implementations, and verification logic.
Introduction to NeMo Gym#
Understand NeMo Gym’s purpose and core components before diving into tutorials.
Motivation and benefits of NeMo Gym.
Core components, configuration, verification and RL terminology.
Understand how NeMo Gym fits within the RL environment ecosystem.
Get Started#
Install and run NeMo Gym to start collecting rollouts.
Install, start servers, and collect your first rollouts in one page.
Step-by-step installation with requirements, configuration, and troubleshooting.
Generate batches of scored interactions and view them with the rollout viewer.
Environment Configuration#
Configure and customize environment components and prepare datasets.
Configure LLM inference backends including vLLM.
Prepare and validate training datasets.
Environment Tutorials#
Learn how to build custom training environments for various RL scenarios.
Build a complete training environment from scratch.
Run multiple training environments simultaneously for rollout collection.
Training Tutorials#
Train models using NeMo Gym with your preferred RL framework.
Hands-on tutorials with NeMo RL, TRL, Unsloth, and more.
Transform rollouts into SFT and DPO format.
Infrastructure#
Deploy NeMo Gym and plan cluster resources for training.
Production deployment patterns and configurations.
Contribute#
Contribute to NeMo Gym development.
Contribute new environments or integrate existing benchmarks.
Implement NeMo Gym integration into a new training framework.