NeMo Gym Documentation#
NeMo Gym is a library for building reinforcement learning (RL) training environments for large language models (LLMs). NeMo Gym provides infrastructure to develop environments, scale rollout collection, and integrate seamlessly with your preferred training framework.
A training environment consists of three server components: Agents orchestrate the rollout lifecycle—calling models, executing tool calls through resources, and coordinating verification. Models provide stateless text generation using LLM inference endpoints. Resources define tasks, tool implementations, and verification logic.
Introduction to NeMo Gym#
Understand NeMo Gym’s purpose and core components before diving into tutorials.
Motivation and benefits of NeMo Gym.
Core components, configuration, verification and RL terminology.
Understand how NeMo Gym fits within the NVIDIA NeMo Framework.
Get Started#
Install and run NeMo Gym to start collecting rollouts.
Run a training environment and start collecting rollouts in under 5 minutes.
Detailed walkthrough of running your first training environment.
Collect and view rollouts
Tutorials#
Hands-on tutorials to build and customize your training environments.
Implement or integrate existing tools and define task verification logic.
Transform rollouts into training data for supervised fine-tuning (SFT) and direct preference optimization (DPO).
Learn how to set up NeMo Gym and NeMo RL training environments, run tests, prepare data, and launch single-node and multi-node training runs.
Contribute#
Contribute to NeMo Gym development.
Contribute new environments or integrate existing benchmarks.
Implement NeMo Gym integration into a new training framework.