Drive a Remote Agent

The remote_agent server lets an agent that runs as its own HTTP service — in your repo, on your infrastructure — be driven by standard rollout collection. Your service implements an endpoint compliant with the OpenAI /v1/responses contract, and the two servers compose as Responses-speaking agents: each call your service receives the conversation so far and returns what it wants to do next. Gym runs the loop, executes environment tools, holds the session, and verifies. Your results land in the standard artifacts (gym eval profile, aggregation, and training pipelines all work unchanged).

collector ──/run──▶ remote_agent (Gym) ──POST /v1/responses──▶ your service ⟲ your model,
                        │      ▲                                               your own tools
                        │      └── Gym-tool results appended,     (any number of internal steps,
                        │          loop repeats                    then: Gym-tool asks
                        ▼                                          or a final answer)
                  resources server (seed / Gym tools / verify — never exposed to your service)

Who does what

Gym is responsible for:

1. Seeding a fresh environment session per rollout and holding its cookies — session state and the task’s answer key (verifier_metadata) never reach your service.
2. Driving the loop: calling your service, executing the tool calls it asks for against the environment, appending the results, and calling again.
3. All failure handling: bounded connect retries, per-call and whole-rollout timeouts, and converting every failure into a reward-0 row in the failures sidecar (a bad rollout never crashes the collection run).
4. Verifying the finished trajectory and reporting the reward.

Your service is responsible for:

1. Implementing POST {agent_base_url}/v1/responses and answering every call with a valid Responses API object (required fields and types are enforced, unknown extra fields are tolerated; an invalid object is a terminal, non-retried failure).
2. Deciding what to do next on each call. Within a call your service can do anything — run its own model for any number of turns, execute its own tools, spawn sub-agents. There are exactly two reasons to return: you need a Gym-hosted tool executed (return the ask), or the rollout is finished (return the answer).
3. Being callable N times per rollout. Each call carries the full conversation so far, so a stateless service needs nothing extra; if you want per-rollout state, set a cookie — Gym echoes your cookies back on every subsequent call of the same rollout.
4. Reporting usage (or omitting it entirely — allowed, but your token metrics are empty).

The contract, exactly

Every request you receive is the task’s responses_create_params with the conversation accumulated in input:

1. input — the task messages, plus (from turn 2 onward) everything so far: your previous output items and one function_call_output per tool call Gym executed for you.
2. tools — the tool schemas this environment serves, verbatim from the dataset row. These are the only tools you may ask Gym to execute.
3. Your own cookies from earlier calls of this rollout, echoed back.

Every response you return is one Responses API object whose output decides the next step. Returning does not mean your agent is done thinking — it means one of two things: “I need a Gym tool” or “I’m finished.” Everything your agent did internally since the last call (its own model turns, its own tool executions) either stays private or rides along as paired call+output records:

Expectations and edge semantics:

1. Ask only for tools the request’s tools declared. An unknown tool name is not an error — Gym sends the resources server’s error text (e.g. a 404) back to you as that call’s function_call_output, and the rollout continues.
2. Malformed arguments (not valid JSON) likewise come back to you as an error output, never a crash.
3. Never return an unpaired function_call for a tool you already executed yourself — unpaired means “Gym, run this.”
4. One rollout = one conversation. Requests of the same rollout share your cookies; different rollouts (including retries of the same task) start clean.

Quickstart

A complete, realistic service against the in-repo stateful counter environment (example_session_state_mgmt; tasks read “add 1 then add 2 then get the count”). The agent brain is the Claude Code CLI: each call, the service renders the conversation into a prompt, lets Claude decide, and translates the decision into tool asks or a final answer. Requirements on the service’s machine: claude on PATH and authenticated (a logged-in CLI or ANTHROPIC_API_KEY in the environment); pick the model with CLAUDE_MODEL (default haiku).

1
# service.py — the Claude Code CLI as the agent's brain
2
import json, os, subprocess
3
from fastapi import FastAPI
4
5
app = FastAPI()
6
MODEL = os.environ.get("CLAUDE_MODEL", "haiku")
7
8
PROMPT = """You are an agent solving a task by calling tools. You do not execute tools \
9
yourself; you ask for them and the results come back in the conversation.
10
11
Task conversation so far:
12
{conversation}
13
14
Tools you may ask for (JSON schemas):
15
{tools}
16
17
Reply with ONLY one JSON object, no prose, no code fences:
18
- to call tools: {{"tool_calls": [{{"name": "<tool>", "arguments": {{...}}}}]}}
19
- to finish:     {{"final": "<answer text>"}}
20
Finish once the conversation contains enough tool results to answer; the final answer must be \
21
exactly what the task asks for.
22
"""
23
24
25
def render(items: list) -> str:
26
    lines = []
27
    for item in items:
28
        kind = item.get("type", "message")
29
        if kind == "message":
30
            content = item.get("content")
31
            if isinstance(content, list):
32
                content = " ".join(c.get("text", "") for c in content)
33
            lines.append(f"[{item.get('role', 'user')}] {content}")
34
        elif kind == "function_call":
35
            lines.append(f"[you asked for] {item['name']}({item.get('arguments')})")
36
        elif kind == "function_call_output":
37
            lines.append(f"[tool result] {item.get('output')}")
38
    return "\n".join(lines)
39
40
41
@app.post("/v1/responses")
42
def responses(params: dict):  # sync handler: FastAPI runs it in a threadpool, so
43
    # concurrent rollouts don't block each other on the subprocess below.
44
    prompt = PROMPT.format(conversation=render(params["input"]), tools=json.dumps(params.get("tools", [])))
45
    # `claude -p` runs Claude's full agentic loop in one invocation: read-only file tools
46
    # (Read/Glob/Grep) work by default; permission-gated tools (WebSearch, Bash, ...) are
47
    # auto-denied headlessly unless pre-approved, e.g. --allowedTools "WebSearch,Bash".
48
    # Either way Claude uses its OWN tools multi-step inside this single call — returning
49
    # to Gym is only for Gym-hosted tools or the final answer.
50
    proc = subprocess.run(
51
        ["claude", "-p", prompt, "--output-format", "json", "--model", MODEL],
52
        capture_output=True, timeout=300, check=True,
53
    )
54
    # The CLI's print mode returns its own JSON envelope (NOT the Anthropic Messages format):
55
    # the final text sits in "result". The Responses object Gym expects is built by hand below —
56
    # translating whatever your brain speaks into Responses format is the service's job.
57
    payload = json.loads(proc.stdout)
58
    decision = json.loads(payload["result"].strip().strip("`").removeprefix("json").strip())
59
    turn = sum(1 for i in params["input"] if i.get("type") == "function_call_output")
60
61
    if "tool_calls" in decision:  # unpaired asks: Gym executes these and calls us again
62
        output = [{"type": "function_call", "id": f"fc_{turn}_{k}", "call_id": f"c_{turn}_{k}",
63
                   "name": call["name"], "arguments": json.dumps(call.get("arguments", {}))}
64
                  for k, call in enumerate(decision["tool_calls"])]
65
    else:  # a final assistant message ends the rollout
66
        output = [{"type": "message", "role": "assistant", "status": "completed", "id": f"m_{turn}",
67
                   "content": [{"type": "output_text", "text": str(decision["final"]), "annotations": []}]}]
68
69
    usage = payload.get("usage") or {}
70
    return {
71
        "id": "claude-cli-service", "created_at": 0.0, "model": f"claude-{MODEL}", "object": "response",
72
        "output": output,
73
        "parallel_tool_calls": False, "tools": [], "tool_choice": "auto",
74
        "usage": {
75
            "input_tokens": usage.get("input_tokens", 0),
76
            "output_tokens": usage.get("output_tokens", 0),
77
            "total_tokens": usage.get("input_tokens", 0) + usage.get("output_tokens", 0),
78
            "input_tokens_details": {"cached_tokens": 0},
79
            "output_tokens_details": {"reasoning_tokens": 0},
80
        },
81
    }

One config file wires the environment and the agent together (this exact shape drove the live end-to-end run — the only thing that changes for your own benchmark is the resources-server block and the ref name):

1
# counter_remote.yaml
2
example_session_state_mgmt_resources_server:
3
  resources_servers:
4
    example_session_state_mgmt:
5
      entrypoint: app.py
6
      domain: agent
7
8
remote_agent:
9
  responses_api_agents:
10
    remote_agent:
11
      entrypoint: app.py
12
      # Your service's address — typically another machine, a container, or a cloud box.
13
      # (For a first try with everything on one machine, http://localhost:9000 works too.)
14
      agent_base_url: http://your-agent-host:9000
15
      resources_server:
16
        type: resources_servers
17
        name: example_session_state_mgmt_resources_server   # the top-level key above
18
      concurrency: 4
19
      max_steps: 8

Run the three pieces in separate terminals — the first two are long-running servers, the third is the driver:

$
# Terminal 1 — on your service's machine:
>
uvicorn service:app --host 0.0.0.0 --port 9000

$
# Terminal 2 — on the Gym machine: environment + remote_agent (leave running):
$
gym env start "+config_paths=[counter_remote.yaml]"

$
# Terminal 3 — on the Gym machine, once the servers report ready:
$
gym eval run --no-serve +agent_name=remote_agent \
>
    +input_jsonl_fpath=resources_servers/example_session_state_mgmt/data/example.jsonl \
>
    +output_jsonl_fpath=results/rollouts.jsonl

Expected: 5 rollouts, mean/reward: 1.0, and each trajectory reads function_call → function_call_output → ... → message. A live end-to-end run of exactly this setup — the service in a container on its own network, Claude deciding every call — scored 5/5.

Configuration knobs

Failure handling and resume

Failures never crash a collection run. A down service (3 connection attempts, then fail), a timed-out call, a malformed reply, or a verifier error becomes a reward-0 row with _ng_failure_class: "remote_agent_error" in the failures sidecar (for results/rollouts.jsonl the sidecar is results/rollouts_failures.jsonl) — the main rollouts file stays clean, and +resume_from_cache=true retries failed tasks up to the attempt cap (NEMO_GYM_MAX_ROLLOUT_ATTEMPTS, default 3; terminal failures — an invalid response shape — are not retried). Connection failures and timeouts name the failing URL or the timeout knob involved.

Gotchas

• Your service is called multiple times per rollout. Design for it: everything you need is in each request’s input, or in your own cookies.
• Redirects are rejected, not followed. Any 3xx from your service fails the rollout with the Location shown — point agent_base_url at the final address. (Followed, a 301/302/303 would silently re-issue the POST as a body-less GET; a 307/308 would silently re-send the task to an address you never configured.)
• Partial usage fails validation. Report the full object — input_tokens, output_tokens, total_tokens, input_tokens_details: {cached_tokens}, output_tokens_details: {reasoning_tokens} — or omit usage entirely (allowed; token metrics are then empty, with a warning).
• A previous run’s output can be reused as the input dataset (for example, re-running the tasks in a failures sidecar). Output rows carry the old run’s results (reward, response, error, internal _ng_* flags); those fields are stripped from each row on the way in, so a stale result can’t collide with or leak into the new run’s output.
• Exposing your service on the public internet needs network-level trust. Gym sends no credential on the /v1/responses call (agent_base_url rejects embedded credentials), so an internet-exposed service should sit behind a VPN, private network, IP allowlist, or authenticating tunnel. Note that proxies configured via HTTP_PROXY/HTTPS_PROXY environment variables are ignored (Gym’s HTTP client does not read them), so a host that can only reach the internet through such a proxy cannot reach your service.