Detailed Setup Guide

Requirements

Hardware Requirements

NeMo Gym is designed to run on standard development machines without specialized hardware:

• GPU: Not required for NeMo Gym library operation
  • GPU may be needed for specific resources servers or model inference (see individual server documentation). E.g. if you are intending to train your model with NeMo-RL, GPU resources are required (see training documentation)
• CPU: Any modern x86_64 or ARM64 processor (e.g., Intel, AMD, Apple Silicon)
• RAM: Minimum 8 GB (16 GB+ recommended for larger environments and datasets)
• Storage: Minimum 2 GB free disk space for installation and basic usage

Software Requirements

• Operating System:
  • Linux (Ubuntu 20.04+, CentOS 7+, or equivalent)
  • macOS (11.0+ for x86_64, 12.0+ for Apple Silicon)
  • Windows via WSL2 (Ubuntu 20.04+ recommended)
• Python: 3.12 or higher (required)
• Git: For cloning the repository
• curl or wget: For installing the UV package manager
• Internet Connection: Required for:
  • Downloading dependencies
  • Accessing model APIs (OpenAI, Azure, etc.)
  • Downloading datasets

Additional Requirements

• API Keys: Model provider access
  • OpenAI API key with available credits (for quickstart and most examples)
  • OR Azure OpenAI credentials
  • OR self-hosted model setup (via vLLM or compatible inference server)
• Ray: Automatically installed as a dependency for distributed processing (no separate setup required)

Verified Configurations

The following configurations have been tested and verified:

Prerequisites

Make sure you have these prerequisites ready before beginning:

• Git (for cloning the repository)
• OpenAI API key with available credits (for the tutorial agent)

1. Clone and Install

Clone the NeMo Gym repository and install dependencies:

$
# Clone the repository
$
git clone git@github.com:NVIDIA-NeMo/Gym.git
$
cd Gym

$
# Clone the repository (if you don't have SSH keys configured)
>
git clone https://github.com/NVIDIA-NeMo/Gym.git
>
cd Gym

$
# Install UV (Python package manager)
$
curl -LsSf https://astral.sh/uv/install.sh | sh
$
source $HOME/.local/bin/env
$
$
# Create virtual environment
$
uv venv --python 3.12
$
source .venv/bin/activate
$
$
# Install NeMo Gym
$
uv sync --extra dev --group docs

✅ Success Check: Verify that you can see something that indicates a newly activated environment such as (.venv) or (NeMo-Gym) in your terminal prompt.

2. Configure Your API Key

Create an env.yaml file in the project root to configure your Policy Model credentials:

$
# Create env.yaml with your OpenAI credentials
$
cat > env.yaml << EOF
$
policy_base_url: https://api.openai.com/v1
$
policy_api_key: sk-your-actual-openai-api-key-here
$
policy_model_name: gpt-4.1-2025-04-14
$
EOF

1
policy_base_url: https://api.openai.com/v1
2
policy_api_key: sk-your-actual-openai-api-key-here
3
policy_model_name: gpt-4.1-2025-04-14

$
python -c "
>
from openai import OpenAI
>
from nemo_gym.global_config import get_global_config_dict
>
>
global_config = get_global_config_dict()
>
>
# Test API access
>
client = OpenAI(
>
    api_key=global_config['policy_api_key'],
>
    base_url=global_config['policy_base_url']
>
)
>
>
# Try a simple request
>
response = client.chat.completions.create(
>
    model=global_config['policy_model_name'],
>
    messages=[{'role': 'user', 'content': 'Say hello'}],
>
    max_tokens=10
>
)
>
print('✅ API key validated successfully!')
>
print(f'Model: {global_config[\"policy_model_name\"]}')
>
print(f'Response: {response.choices[0].message.content}')
>
"

The cost for running rollouts using the OpenAI API can be calculated using the following rough formula: per token API cost × average number of input/output tokens × num_repeats × limit.

• Per token API cost: See the OpenAI API pricing for more details https://openai.com/api/pricing/.
• Average number of input/output tokens: After rollouts are run, you can see the input/output token usage in the returned response.
• Num repeats and limit: These parameters are set in the rollout collection command later.

3. Start the Servers

$
# Define which servers to start
$
config_paths="resources_servers/example_single_tool_call/configs/example_single_tool_call.yaml,\
>
responses_api_models/openai_model/configs/openai_model.yaml"
$
$
# Start all servers
$
ng_run "+config_paths=[${config_paths}]"

✅ Success Check: Verify that you can see output like:

INFO:     Started server process [12345]
INFO:     Uvicorn running on http://127.0.0.1:11000 (Press CTRL+C to quit)
INFO:     Started server process [12346]  
INFO:     Uvicorn running on http://127.0.0.1:62920 (Press CTRL+C to quit)
...

$
curl http://localhost:11000/server_instances

When you ran ng_run, it started all the servers you configured:

• Head server: coordinating all components
• Resources server: defining tools and verification
• Model server: providing LLM inference
• Agent server: orchestrating how the model interacts with the resources. The agent server calls the model and resources servers using REST requests.

$
source .venv/bin/activate

4. Test the Setup

Open a new terminal (keep servers running in the first one).

$
# Navigate to project directory
$
cd /path/to/Gym
$
$
# Activate virtual environment
$
source .venv/bin/activate
$
$
# Test the agent
$
python responses_api_agents/simple_agent/client.py

✅ Success Check: Verify that you can see JSON output showing:

1. Agent calling the weather tool
2. Weather tool returning data
3. Agent responding to the user

Example output:

1
[
2
    {
3
        "arguments": "{\"city\":\"San Francisco\"}",
4
        "call_id": "call_OnWAk719Jr3tte4OmCJtJOB4",
5
        "name": "get_weather",
6
        "type": "function_call",
7
        "id": "fc_68a3739f2f0081a1aae4b93d5df07c100cb216b5cc4adbc4",
8
        "status": "completed"
9
    },
10
    {
11
        "call_id": "call_OnWAk719Jr3tte4OmCJtJOB4",
12
        "output": "{\"city\": \"San Francisco\", \"weather_description\": \"The weather in San Francisco is cold.\"}",
13
        "type": "function_call_output",
14
        "id": null,
15
        "status": null
16
    },
17
    {
18
        "id": "msg_68a373a1099081a1bb265ecf3b26c0dc0cb216b5cc4adbc4",
19
        "content": [
20
            {
21
                "annotations": [],
22
                "text": "The weather in San Francisco tonight is cold. You might want to wear layers or bring a jacket to stay comfortable while you're out. Let me know if you want outfit advice or tips on where to go!",
23
                "type": "output_text",
24
                "logprobs": []
25
            }
26
        ],
27
        "role": "assistant",
28
        "status": "completed",
29
        "type": "message"
30
    }
31
]

Error code: 429 - You exceeded your current quota

Error code: 401 - Incorrect API key provided

Error code: 404 - Model not found

$
# Quick test to verify API access using your env.yaml credentials
$
python -c "
>
from openai import OpenAI
>
from nemo_gym.global_config import get_global_config_dict
>
>
config = get_global_config_dict()
>
client = OpenAI(
>
    api_key=config['policy_api_key'],
>
    base_url=config['policy_base_url']
>
)
>
print('✅ API access verified:', client.models.list().data[0].id)
>
"

File Structure After Setup

Your directory should look like this:

$
Gym/
$
├── env.yaml                    # Your API credentials (git-ignored)
$
├── .venv/                      # Virtual environment (git-ignored)
$
├── nemo_gym/                   # Core library code
$
├── resources_servers/          # Tools and environments
$
├── responses_api_models/       # Model integrations  
$
├── responses_api_agents/       # Agent implementations
$
└── docs/                       # Documentation files

Next Steps

You’ve confirmed that NeMo Gym is working — the agent can call tools and return results. But a single interaction isn’t enough for RL training. The next step is to collect batches of scored interactions (rollouts) that become your training data.