About

Ecosystem

View as Markdown

We’re building NeMo Gym to integrate with a broad set of RL training frameworks and environment libraries.

We would love your contribution! Open a PR to add an integration, or file an issue to share what would be valuable for you.

Training Framework Integrations

We have hands-on tutorials with supported training frameworks to help you train with NeMo Gym environments. If you’re interested in integrating another training framework, see the Training Framework Integration Guide.

  • NeMo RL - GRPO training to improve multi-step tool calling on the Workplace Assistant environment
  • OpenRLHF - example agent executor for RL training
  • Unsloth - GRPO training on instruction following and reasoning environments
  • NeMo Customizer - (In progress)
  • VeRL - (In progress)

Environment Library Integrations

NeMo Gym integrates with external environment libraries and benchmarks. See the README for the full list—here are a few examples:

  • Aviary - environments spanning math, knowledge, biological sequences, scientific literature search, and protein stability
  • Harbor - popular agentic environments including Terminus2
  • OpenEnv - open environment interface via MCP
  • Reasoning Gym - reasoning environments spanning computation, cognition, logic and more
  • Verifiers - environments spanning coding, data & ML, science & reasoning, tool use and more
  • BrowserGym - (In progress) - environments for web task automation

Agent Harness Integrations

Popular agent harnesses are available out of the box for evaluation and training.

Agent Framework Integrations

Use your custom agent harnesses with NeMo Gym environments.

  • LangGraph - build custom graph-based agent workflows

NeMo Gym is a component of NVIDIA NeMo, a GPU-accelerated platform for building and training generative AI models.

Depending on your workflow, you may also find these libraries useful:

LibraryPurpose
NeMo Megatron-BridgePretraining and fine-tuning with Megatron-Core
NeMo AutoModelPyTorch native training for Hugging Face models
NeMo RLScalable post-training with GRPO, DPO, and SFT
NeMo GymRL environment infrastructure and rollout collection (this project)
NeMo CuratorData preprocessing and curation
NeMo Data DesignerHigh-quality Synthetic data generation from scratch or seed data
NeMo EvaluatorModel evaluation and benchmarking
NeMo GuardrailsProgrammable safety guardrails
NeMo SkillsConvenience pipelines used by LLM researchers across SDG, evaluation and training