Configure Agents | NeMo Gym

The Agent Server hosts the agent harness, which calls the model and routes tool calls. It does not run an LLM itself; it is orchestration code, and the harness uses a Model Server or calls an inference endpoint directly. Support for multiple tool calls per turn and for multi-turn conversations depends on the harness, its Gym integration, and the Environment Server. An Environment Server either orders the episode’s calls or relays the whole episode to the agent, as in the pseudocode below.

Rollout Lifecycle

The following pseudocode illustrates a typical agent rollout in three phases: initialize the episode, run the agent loop, and grade the result. During the agent loop, the agent sends the conversation to the model, gets back a response, and if the model makes any tool calls, it routes them to the Resources server and feeds the results back to the model. The loop repeats until stop criteria are met, such as model max sequence length or the agent reaching a defined max steps or turns. Once the loop completes, the agent calls the Resources server to verify the result and collect a reward.

# Agent Server - pseudocode
class Agent:
    async def run(self, task_data):
        # 1. Initialize episode
        resources_server.seed_session(task_data)
        # 2. Run the agent loop
        response = self.responses(task_data.prompt, task_data.tools)
        # 3. Grade the result
        reward = resources_server.verify(response, task_data.ground_truth)
        return response, reward
    async def responses(self, prompt, tools):
        conversation = prompt
        step = 0
        # Agent loop
        while step < max_steps:
            model_output = model_server.responses(conversation, tools)
            conversation.append(model_output)
            if model_output is text:
                break # model is done, no more tool calls
            for tool_call in model_output.function_calls:
                result = resources_server.post(f"/{tool_call.name}", tool_call.arguments)
                conversation.append(result)
            step += 1
        return conversation

The pseudocode is the legacy_agent path: rollout collection reaches the agent through a legacy_agent Environment Server that relays /run unchanged, and the agent runs the whole episode itself. Most agents, including Remote Agent, still run this way. Agents that implement agent sessions, such as SimpleAgent and the sandboxed harnesses, can instead run under an Environment Server that drives agent sessions, such as single_agent_turn. Most shipped SimpleAgent configurations do, through its flat-row adapter single_agent_turn_legacy. Such a server calls seed_session and verify on the Resources server itself and drives the agent through session endpoints, so the agent does not decide when to verify. The harbor Environment Server drives sessions the same way for task folders, which need no Resources server. See Design Agent-Environment Interactions.

Choose a Path

Start from what you have, and move to a deeper path only when you need what it adds.

Path	What you do	What Gym does	Limits
Integrate Existing Agents	Pick an agent that Gym already integrates. Pair it with a benchmark in YAML or, for Harbor task folders, pass it to `gym eval run` as `--agent`.	Runs the episode, captures model calls and observations, and verifies.	Each agent has its own supported benchmarks, tools, and training status. Check its row in the catalog.
Remote Agent	An HTTP service that you host, with a `/v1/responses` endpoint that returns the agent’s next step.	Drives the loop, runs the environment tools that your service requests, holds the session, and verifies the result.	Scores only. No model-call capture and no training, because your service calls its own model and Gym sees only what the service returns. No task sandbox, because the service cannot reach one.
Agent Server adapter	An adapter under `responses_api_agents/<harness>/` that runs the harness or framework.	Everything in What Gym provides, including model-call capture, tool grants, and task sandboxes.	You write the conversion and mapping code described in What you write.
Custom Environment Server	A server that owns the episode protocol, such as `nemo_user_sim`, which routes simulated-user and assistant turns through their own Agent Servers.	Rollout collection, the `BaseEnvironmentServer` base class with concurrency and cleanup handling, and the Resources Server session and verification APIs.	Not covered on this page. See Design Agent-Environment Interactions to choose a shipped Environment Server or write your own, and system design for how Environment Servers drive rollouts.

Grow from a score to training

A common route for an agent that you already run:

Score it. Drive the agent through a Remote Agent and get verifier rewards without rewriting it. Gym treats the agent as a black box.
Evaluate it with Gym’s models, tools, and sandboxes. Build an Agent Server adapter so that Gym can show its model calls, tool use, and execution trace. These rollouts are not yet validated for training.
Train it. Confirm that every policy-generating model call is captured with token IDs and logprobs, then run an end-to-end training job. This needs a model server that returns token information, such as the vLLM model server. See Choose the capabilities you need.

Server Configuration

To add conversations, tool timing, and model-call references to a custom harness, see Rollout Evidence.

An Agent Server runs with num_workers above 1 when its entrypoint module defines the module-level app that each Uvicorn worker serves, the same way as a Resources Server. An agent that cannot run in several processes overrides multiple_workers_unsupported_reason() instead, and refuses several workers at startup. Concurrency limits such as an agent’s concurrency semaphore apply per worker, so the server as a whole allows num_workers times the configured limit.

Agent Server Fields for server configuration syntax and fields.