RL Framework Compatibility

Reference for NeMo Gym version compatibility with supported training frameworks.

NeMo RL Container

The following table maps NeMo Gym versions to compatible NeMo RL containers. Use the latest version when possible; the table provides historical compatibility for users who cannot upgrade.

NeMo Gym Version	Container / Docker File	Recipe
v0.1.1	`nvcr.io/nvidia/nemo-rl:v0.4.0.nemotron_3_nano`	Nemotron 3 Nano
v0.2.0	`nvcr.io/nvidia/nemo-rl:v0.5.0.nemotron_3_super`	Nemotron 3 Super
v0.3.0	Dockerfile	Nemotron 3 Ultra

NeMo RL GRPO for the NeMo RL training tutorial.

Unsloth

The NeMo Gym integration with Unsloth is tested on unsloth==2026.1.4 and unsloth_zoo==2026.1.4. Other versions are not guaranteed to work.

Unsloth for the Unsloth training tutorial.

VeRL

NeMo Gym 0.2.1+ is compatible with verl pinned to the commit in REQUIRED_VERL.txt, tested on the verlai/verl:vllm017.latest container (vLLM 0.17.0). Other versions are not guaranteed to work.

Training with VeRL for the verl training tutorial.