> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For full documentation content, see https://docs.nvidia.com/nemo/gym/llms-full.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# Integrate RL Frameworks

These guides cover how to integrate NeMo Gym into a new RL training framework. Use them if you are:- A training framework maintainer adding NeMo Gym support
- Contributing NeMo Gym integration for a training framework that does not have one yetJust want to train models? See [Training Tutorials](/training-tutorials) for supported frameworks.## PrerequisitesBefore integrating Gym into your training framework, ensure you have:- An RL training framework with policy optimization support (PPO, GRPO, or similar)
- A generation backend (vLLM, SGLang, or equivalent)
- Familiarity with OpenAI-compatible HTTP server APIs## Integration ComponentsGym integration requires implementing the following components in your training framework:OpenAI-compatible HTTP server requirements and existing implementations across RL frameworks.prerequisiteFixes for on-policy training in multi-step and multi-turn scenarios to prevent train-generation mismatch.prerequisiteImplementation components, form factor, and reference implementations from NeMo RL.implementationValidation criteria and benchmarks to verify correct Gym integration.validation## Integration WorkflowThe typical integration workflow follows this sequence:| Step | Component             | Description                                                                                |
| ---- | --------------------- | ------------------------------------------------------------------------------------------ |
| 1    | Generation backend    | Expose your generation engine, such as vLLM or SGLang, as an OpenAI-compatible HTTP server |
| 2    | On-policy corrections | Implement token ID fixes to prevent re-tokenization and re-templating issues               |
| 3    | Gym integration       | Connect Gym to your training loop using the rollout orchestration APIs                     |
| 4    | Validation            | Verify integration using the success criteria benchmarks                                   |