Reference

RL Framework Compatibility

View as Markdown

Reference for NeMo Gym version compatibility with supported training frameworks.

NeMo RL Container

The following table shows which NeMo Gym version each published NeMo RL container bundles and the model recipe that container supports. NeMo Gym and NeMo RL release independently, so not every NeMo Gym version has a matching NeMo RL image.

Single source of truth: When a tutorial or install page needs a container, use the value from this table for the relevant Gym version and recipe. Do not hard-code a different tag elsewhere without a comment pointing back here.

v0.3.0 exception: NeMo Gym v0.3.0 paired with Nemotron 3 Ultra has no pre-built NGC container that packaged Gym, so this row references the ultra-v3 Dockerfile. That is a one-off.

The latest published pairing is NeMo RL v0.7.0 with NeMo Gym v0.4.0.

Match the container to your model recipe, not only your NeMo Gym version. For example, the NeMo RL GRPO tutorial trains Nemotron Nano 9B v2 and uses the Nano NGC container from the v0.1.1 row below—not the v0.3.0 Ultra Dockerfile.

For end-to-end setup and training instructions, see the NeMo RL GRPO tutorial.

Unsloth

The NeMo Gym integration with Unsloth is tested on unsloth==2026.1.4 and unsloth_zoo==2026.1.4. Other versions are not guaranteed to work.

For setup and training instructions, see the Unsloth tutorial.

VeRL

NeMo Gym 0.2.1+ is compatible with verl pinned to the commit in REQUIRED_VERL.txt, tested on the verlai/verl:vllm017.latest container (vLLM 0.17.0). Other versions are not guaranteed to work.

For setup and training instructions, see the Training with VeRL tutorial.