Gym Integration

Serve NEL benchmarks for NeMo Gym training and consume remote Gym environments for evaluation.

Architecture

There are three integration modes:

Set your model endpoint once before running the nel eval run examples:

export NEMO_MODEL_URL="https://integrate.api.nvidia.com/v1/chat/completions"
export NEMO_MODEL_ID="nvidia/nemotron-3-super-120b-a12b"
export NVIDIA_API_KEY="your-api-key-here"

Mode 1: Serve for Gym Training

Start the environment server

nel serve -b gsm8k -p 9090

The server speaks Gym’s native protocol:

• POST /seed_session — returns prompt and expected answer
• POST /verify — accepts response, returns {reward: float}
• GET /health — health check
• GET /dataset_size — number of problems

Point Gym at it

In your Gym training config:

resource_servers:
  nemo_evaluator:
    endpoint: http://evaluator-host:9090
    eval_type: gsm8k

Get decision-grade scores after training

The same server also speaks NEL’s enriched protocol:

nel eval run --bench gym://evaluator-host:9090 --repeats 4

This produces the full artifact suite (trajectories, CI, failure analysis) from the same environment.

Mode 2: Export for ng_collect_rollouts

For batch rollout collection without a live server:

nel serve -b gsm8k --export-data /tmp/evaluator_data

Or via Python:

import asyncio
import nemo_evaluator.benchmarks  # noqa: F401
from nemo_evaluator import get_environment
async def main():
    env = get_environment("gsm8k")
    import json
    with open("/tmp/rollout_data.jsonl", "w") as f:
        for idx in range(len(env)):
            seed = await env.seed(idx)
            f.write(json.dumps({
                "responses_create_params": {"input": seed.messages or [{"role": "user", "content": seed.prompt}]},
                "expected_answer": seed.expected_answer,
                "uuid": f"gsm8k-{idx}",
                "metadata": seed.metadata,
            }) + "\n")
asyncio.run(main())

Mode 3: Consume a Remote Environment

Evaluate a model against any running nel serve endpoint using GymEnvironment:

nel eval run --bench gym://localhost:9090 --repeats 2
nel eval run --bench gym://gym-cluster:8080 --repeats 4 --output-dir ./results/remote

Because NEL makes the model call (not the environment server), you get full observability: per-request latency, token counts, reasoning tokens, failure categorization.

Managed Gym Environments

For environments that need a server started and stopped automatically, use gym:// with a benchmark name (not host:port). The registry auto-detects that it’s a name and starts a managed server:

nel eval run --bench gym://gsm8k --repeats 4

Or with a custom server command:

nel eval run --bench "gym://cmd:python my_server.py" --repeats 4

The server is started automatically, health-checked, used for evaluation, and torn down on completion.

Python API

import asyncio
from nemo_evaluator.environments.gym import GymEnvironment
from nemo_evaluator import run_evaluation, ChatSolver, ModelClient
async def main():
    env = GymEnvironment("http://localhost:9090")
    client = ModelClient(base_url="https://api.example.com/v1", model="my-model")
    solver = ChatSolver(client)
    bundle = await run_evaluation(env, solver, n_repeats=4)
    return bundle
asyncio.run(main())