Gym Integration

Serve NEL benchmarks for NeMo Gym training and consume remote Gym environments for evaluation.

Architecture

There are three integration modes:

Set your model endpoint once before running the nel eval run examples:

$
export NEMO_MODEL_URL="https://integrate.api.nvidia.com/v1/chat/completions"
$
export NEMO_MODEL_ID="nvidia/nemotron-3-super-120b-a12b"
$
export NVIDIA_API_KEY="your-api-key-here"

Mode 1: Serve for Gym Training

Start the environment server

$
nel serve -b gsm8k -p 9090

The server speaks Gym’s native protocol:

• POST /seed_session — returns prompt and expected answer
• POST /verify — accepts response, returns {reward: float}
• GET /health — health check
• GET /dataset_size — number of problems

Point Gym at it

In your Gym training config:

1
resource_servers:
2
  nemo_evaluator:
3
    endpoint: http://evaluator-host:9090
4
    eval_type: gsm8k

Get decision-grade scores after training

The same server also speaks NEL’s enriched protocol:

$
nel eval run --bench gym://evaluator-host:9090 --repeats 4

This produces the full artifact suite (trajectories, CI, failure analysis) from the same environment.

Mode 2: Export for ng_collect_rollouts

For batch rollout collection without a live server:

$
nel serve -b gsm8k --export-data /tmp/evaluator_data

Or via Python:

1
import asyncio
2
import nemo_evaluator.benchmarks  # noqa: F401
3
from nemo_evaluator import get_environment
4
5
async def main():
6
    env = get_environment("gsm8k")
7
8
    import json
9
    with open("/tmp/rollout_data.jsonl", "w") as f:
10
        for idx in range(len(env)):
11
            seed = await env.seed(idx)
12
            f.write(json.dumps({
13
                "responses_create_params": {"input": seed.messages or [{"role": "user", "content": seed.prompt}]},
14
                "expected_answer": seed.expected_answer,
15
                "uuid": f"gsm8k-{idx}",
16
                "metadata": seed.metadata,
17
            }) + "\n")
18
19
asyncio.run(main())

Mode 3: Consume a Remote Environment

Evaluate a model against any running nel serve endpoint using GymEnvironment:

$
nel eval run --bench gym://localhost:9090 --repeats 2
$
nel eval run --bench gym://gym-cluster:8080 --repeats 4 --output-dir ./results/remote

Because NEL makes the model call (not the environment server), you get full observability: per-request latency, token counts, reasoning tokens, failure categorization.

Managed Gym Environments

For environments that need a server started and stopped automatically, use gym:// with a benchmark name (not host:port). The registry auto-detects that it’s a name and starts a managed server:

$
nel eval run --bench gym://gsm8k --repeats 4

Or with a custom server command:

$
nel eval run --bench "gym://cmd:python my_server.py" --repeats 4

The server is started automatically, health-checked, used for evaluation, and torn down on completion.

Python API

1
import asyncio
2
from nemo_evaluator.environments.gym import GymEnvironment
3
from nemo_evaluator import run_evaluation, ChatSolver, ModelClient
4
5
async def main():
6
    env = GymEnvironment("http://localhost:9090")
7
    client = ModelClient(base_url="https://api.example.com/v1", model="my-model")
8
    solver = ChatSolver(client)
9
10
    bundle = await run_evaluation(env, solver, n_repeats=4)
11
    return bundle
12
13
asyncio.run(main())