Agent Server

The Agent server is the central component of environment design. It defines whether a rollout is single-step or multi-step, single-turn or multi-turn, and orchestrates all interaction logic — calling the model, executing tool calls through resources, and collecting the final reward. The Agent server does not run an LLM itself, it is orchestration code that delegates all text generation to the Model server.

Rollout Lifecycle

The following pseudocode illustrates a typical agent rollout in three phases: initialize the episode, run the agent loop, and grade the result. During the agent loop, the agent sends the conversation to the model, gets back a response, and if the model makes any tool calls, it routes them to the Resources server and feeds the results back to the model. The loop repeats until stop criteria are met, such as model max sequence length or the agent reaching a defined max steps or turns. Once the loop completes, the agent calls the Resources server to verify the result and collect a reward.

# Agent Server - pseudocode
class Agent:
    async def run(self, task_data):
        # 1. Initialize episode
        resources_server.seed_session(task_data)

        # 2. Run the agent loop
        response = self.responses(task_data.prompt, task_data.tools)

        # 3. Grade the result
        reward = resources_server.verify(response, task_data.ground_truth)
        return response, reward

    async def responses(self, prompt, tools):
        conversation = prompt
        step = 0

        # Agent loop
        while step < max_steps:
            model_output = model_server.responses(conversation, tools)
            conversation.append(model_output)

            if model_output is text:
                break # model is done, no more tool calls

            for tool_call in model_output.function_calls:
                result = resources_server.post(f"/{tool_call.name}", tool_call.arguments)
                conversation.append(result)

            step += 1

        return conversation

Integrate Existing Agents

You can use an existing agent in NeMo Gym, integrate an external one, or build your own from scratch.

SimpleAgent is a native NeMo Gym agent that handles general-purpose multi-step tool calling with configurable max steps, and works with any Resources server out of the box. NeMo Gym also includes agents that integrate external tools: for example, MiniSWEAgent wraps an external coding harness running in Docker containers and converts its output back into the NeMo Gym format.

Tools in Agent vs. Resources Server

Existing agents may come with predefined tools, allowing you to leverage them directly and use the Resources server to supplement with any additional external tools. When building a new environment, prefer defining tools in the Resources server rather than the Agent server. This separation of concerns allows different agents to share the same Resources server without duplicating tool logic.

Server Configuration

See also

Agent Server Fields for server configuration syntax and fields.