For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
DocumentationAPI Reference
DocumentationAPI Reference
  • Documentation
    • Home
  • About
    • Concepts
    • Ecosystem
  • Get Started
    • Quickstart
    • Detailed Setup Guide
    • Install from PyPI
    • Rollout Collection
  • Agent Server
  • Model Server
    • vLLM
  • Resources Server
  • Data
    • Prepare and Validate
    • Download from Hugging Face
    • Prompt Config
  • Environment Tutorials
    • Single-Step Environment
    • Multi-Step Environment
    • Stateful Environment
    • Real-World Environment
    • Integrate external libraries
    • Aggregate Metrics
    • LLM-as-Judge Verification
  • Benchmarks
    • Run benchmarks
    • Add a benchmark
    • Design a customer evaluation
  • Training Tutorials
    • NeMo RL
    • Unsloth
    • Multi-Environment Training
    • Offline Training (SFT/DPO)
  • Model Recipes
    • Nemotron 3 Nano
    • Nemotron 3 Super
  • Infrastructure
    • Deployment Topology
    • Engineering Notes
  • Reference
    • Configuration
    • RL Framework Compatibility
    • CLI Commands
    • FAQ
  • Troubleshooting
    • Configuration Errors
  • Contribute
    • Development Setup
    • Environments
    • Integrate RL Frameworks
NVIDIANVIDIA
Developer-friendly docs for your API
Privacy Policy | Manage My Privacy | Do Not Sell or Share My Data | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2026, NVIDIA Corporation.

LogoLogoNeMo Gym
On this page
  • NeMo RL Container
  • Unsloth
Reference

RL Framework Compatibility

||View as Markdown|
Previous

Configuration

Next

CLI Commands

Reference for NeMo Gym version compatibility with supported training frameworks.

Ecosystem for training framework integrations and NeMo RL GRPO for training with NeMo Gym.

NeMo RL Container

The following table maps NeMo Gym versions to compatible NeMo RL containers. Use the latest version when possible; the table provides historical compatibility for users who cannot upgrade.

NeMo Gym VersionNeMo RL ContainerRecipe
v0.1.1nvcr.io/nvidia/nemo-rl:v0.4.0.nemotron_3_nanoNemotron 3 Nano
v0.2.0nvcr.io/nvidia/nemo-rl:v0.5.0.nemotron_3_superNemotron 3 Super

Unsloth

The NeMo Gym integration with Unsloth is tested on unsloth==2026.1.4 and unsloth_zoo==2026.1.4. Other versions are not guaranteed to work.

Unsloth for the Unsloth training tutorial.