RL Framework Compatibility
Reference for NeMo Gym version compatibility with supported training frameworks.
NeMo RL Container
The following table maps NeMo Gym versions to compatible NeMo RL containers for each model recipe. Use the latest version when possible; the table provides historical compatibility for users who cannot upgrade.
Single source of truth: When a tutorial or install page needs a container, use the value from this table for the relevant Gym version and recipe. Do not hard-code a different tag elsewhere without a comment pointing back here.
v0.3.0 exception: NeMo Gym v0.3.0 paired with Nemotron 3 Ultra has no pre-built NGC container that packaged Gym, so this row references the ultra-v3 Dockerfile. That is a one-off.
v0.4.0 and later: NeMo RL plans to publish an NGC container for every Gym release. New rows should use NGC tags; Dockerfile references should not be added unless the same gap recurs.
Match the container to your model recipe, not only your NeMo Gym version. For example, the NeMo RL GRPO tutorial trains Nemotron Nano 9B v2 and uses the Nano NGC container from the v0.1.1 row below—not the v0.3.0 Ultra Dockerfile.
NeMo RL GRPO for the NeMo RL training tutorial.
Unsloth
The NeMo Gym integration with Unsloth is tested on unsloth==2026.1.4 and unsloth_zoo==2026.1.4. Other versions are not guaranteed to work.
Unsloth for the Unsloth training tutorial.
VeRL
NeMo Gym 0.2.1+ is compatible with verl pinned to the commit in REQUIRED_VERL.txt, tested on the verlai/verl:vllm017.latest container (vLLM 0.17.0). Other versions are not guaranteed to work.
Training with VeRL for the verl training tutorial.