Quickstart

This is the quickstart—get running in under 5 minutes. For a more detailed walkthrough, see the Detailed Setup Guide.

Before You Start

Make sure you have these prerequisites ready:

Git for cloning the repository
OpenAI API key with available credits (requires ~$0.01-0.10 for all tutorials)

Follow the tabs sequentially to install NeMo Gym, start the servers, and collect your first verified rollouts for RL training.

1. Set Up

2. Start Servers

3. Collect Rollouts

4. Clean Up Servers

Install NeMo Gym

Get NeMo Gym installed and ready to use:

$ # Clone the repository
$ git clone git@github.com:NVIDIA-NeMo/Gym.git
$ cd Gym
$ 
$ # Install UV (Python package manager)
$ curl -LsSf https://astral.sh/uv/install.sh | sh
$ source $HOME/.local/bin/env
$ 
$ # Create virtual environment
$ uv venv --python 3.12
$ source .venv/bin/activate
$ 
$ # Install NeMo Gym
$ uv sync --extra dev --group docs

Configure Your API Key

Create an env.yaml file that contains your OpenAI API key and the Policy Model you want to use. Replace your-openai-api-key with your actual key. This file helps keep your secrets out of version control while still making them available to NeMo Gym.

$ echo "policy_base_url: https://api.openai.com/v1
> policy_api_key: your-openai-api-key
> policy_model_name: gpt-4.1-2025-04-14" > env.yaml

Note: We use GPT-4.1 in this quickstart because it provides low latency (no reasoning step) and works reliably out-of-the-box. NeMo Gym is not limited to OpenAI models—you can use self-hosted models via vLLM or any OpenAI-compatible inference server that supports function calling. Refer to the Detailed Setup Guide for details.

Next Steps

Now that you can generate rollouts, choose your path:

Start Training

Train models using NeMo Gym with your preferred RL framework.

Use an Existing Environment

Explore environments available for training and evaluation.

Build a Custom Environment

Implement or integrate existing tools and define task verification logic.