Configure Agents

The Agent server is the central component of environment design. It defines whether a rollout is single-step or multi-step, single-turn or multi-turn, and orchestrates all interaction logic — calling the model, executing tool calls through resources, and collecting the final reward. The Agent server does not run an LLM itself, it is orchestration code that delegates all text generation to the Model server.

Rollout Lifecycle

The following pseudocode illustrates a typical agent rollout in three phases: initialize the episode, run the agent loop, and grade the result. During the agent loop, the agent sends the conversation to the model, gets back a response, and if the model makes any tool calls, it routes them to the Resources server and feeds the results back to the model. The loop repeats until stop criteria are met, such as model max sequence length or the agent reaching a defined max steps or turns. Once the loop completes, the agent calls the Resources server to verify the result and collect a reward.

1
# Agent Server - pseudocode
2
class Agent:
3
    async def run(self, task_data):
4
        # 1. Initialize episode
5
        resources_server.seed_session(task_data)
6
7
        # 2. Run the agent loop
8
        response = self.responses(task_data.prompt, task_data.tools)
9
10
        # 3. Grade the result
11
        reward = resources_server.verify(response, task_data.ground_truth)
12
        return response, reward
13
14
    async def responses(self, prompt, tools):
15
        conversation = prompt
16
        step = 0
17
18
        # Agent loop
19
        while step < max_steps:
20
            model_output = model_server.responses(conversation, tools)
21
            conversation.append(model_output)
22
23
            if model_output is text:
24
                break # model is done, no more tool calls
25
26
            for tool_call in model_output.function_calls:
27
                result = resources_server.post(f"/{tool_call.name}", tool_call.arguments)
28
                conversation.append(result)
29
30
            step += 1
31
32
        return conversation

Existing Agents

See Integrate Existing Agents to use a built-in NeMo Gym agent, wrap an external agent harness, or decide when tool logic belongs in the Agent server versus the Resources server.

Server Configuration