Resources Server APIs

A Resources Server implements the Environment interface related to: its tools, state, and reward logic.

Resources Server Protocol Landscape

NeMo Gym ships two framework-standard base protocols for coordinating an episode between the Agent and the Environment. Agent behavior and rollout orchestration are implemented by responses() and run() behind the Agent Server’s HTTP endpoints; the Environment is implemented by the Resources Server.

Each protocol has a matching built-in Agent Server: simple_agent for the /seed_session and /verify interface, and gymnasium_agent for the /reset and /step interface.

Pick one base protocol and build the Resources Server and Agent Server to match. GymnasiumServer exposes /reset and /step and does not expose /verify. SimpleResourcesServer defines /seed_session and /verify but can be extended with additional environment-specific routes.

Seed/Verify Protocol Extensions

Several environments keep the /seed_session and /verify lifecycle while adding bespoke transition APIs. These extensions require a matching Agent Server; sharing an endpoint name such as /step does not make their request and response contracts interchangeable.

Some Agent Servers also use variants of the seed/verify sequence—for example, skipping a no-op /seed_session, treating it as optional, or calling /verify without tool execution. These are Agent Server behaviors built around the seed/verify API rather than new Resources Server base classes.

Agent-Embedded Environments

Some Agent Servers embed the environment or an external benchmark harness in their own process. Examples include OSWorld, Tau2, SWE wrappers, Harbor, PinchBench, and Verifiers. Their /run implementation computes reward without calling a Resources Server, so they do not implement a Resources Server episode protocol.

Action and Tool Transports

The episode protocol defines initialization, transitions, and scoring. The action transport separately defines how the Agent affects the Environment during the episode.

Seed/Verify Protocol: seed_session() and verify()

The seed/verify protocol brackets the Agent harness with environment initialization and final verification:

Client → Agent Server: POST /run
Agent Server → Resources Server: POST /seed_session
Agent Server run() → Agent Server responses(): POST /v1/responses
  responses() ↔ Model Server
  responses() → Resources Server tool endpoint
  ... repeat the model/tool Agent loop
Agent Server → Resources Server: POST /verify
Agent Server → Client: completed trajectory and reward

For the built-in simple_agent, run() performs the lifecycle orchestration and responses() implements the Agent loop. The Agent Server exposes those methods as /run and /v1/responses, respectively.

Lifecycle API

SimpleResourcesServer exposes:

1
async def seed_session(
2
    self,
3
    body: BaseSeedSessionRequest,
4
) -> BaseSeedSessionResponse:
5
    ...
6
7
async def verify(
8
    self,
9
    body: BaseVerifyRequest,
10
) -> BaseVerifyResponse:
11
    ...

• body in seed_session() contains fields used to initialize an episode. Define a BaseSeedSessionRequest subclass for fields such as an initial board, task ID, or sandbox image. The default implementation is a no-op.
• body.responses_create_params in verify() is the original model request.
• body.response in verify() is the completed NeMoGymResponse, including the trajectory assembled by the Agent Server.
• Custom task metadata requires a BaseVerifyRequest subclass. Declare fields such as answer or expected_state; undeclared fields are not retained by the base Pydantic model.
• request: Request, when added to either method signature, provides access to the per-rollout session ID.

Direct HTTP Tool-Calling Protocol

The seed/verify episode protocol does not itself define how the Agent acts on the environment. simple_agent supplies a direct HTTP function-tool protocol inside responses():

1. The input row declares function schemas in responses_create_params.tools.
2. The Agent Server sends those schemas to the Model Server.
3. The model returns a structured function_call with name, arguments, and call_id.
4. simple_agent parses arguments and sends POST /{name} to the Resources Server.
5. The Resources Server’s response becomes a function_call_output associated with the same call_id.
6. responses() sends the updated trajectory back to the model and repeats until the model returns a final response or the step limit is reached.

The tool name in the input schema must match the Resources Server route:

1
class SquareRequest(BaseModel):
2
    value: int
3
4
5
class SquareResponse(BaseModel):
6
    result: int
7
8
9
class CalculatorServer(SimpleResourcesServer):
10
    def setup_webserver(self) -> FastAPI:
11
        app = super().setup_webserver()
12
        app.post("/square")(self.square)
13
        return app
14
15
    async def square(self, body: SquareRequest) -> SquareResponse:
16
        return SquareResponse(result=body.value**2)

For a model response such as:

1
{
2
  "type": "function_call",
3
  "name": "square",
4
  "arguments": "{\"value\": 4}",
5
  "call_id": "call_123"
6
}

simple_agent calls POST /square with {"value": 4}. The Agent Server performs this dispatch; the Resources Server only implements the endpoint.

Minimal Seed/Verify Environment

The complete server below combines the tool route with final trajectory verification:

1
from fastapi import FastAPI
2
from pydantic import BaseModel
3
4
from nemo_gym.base_resources_server import (
5
    BaseVerifyRequest,
6
    BaseVerifyResponse,
7
    SimpleResourcesServer,
8
)
9
10
11
class SquareRequest(BaseModel):
12
    value: int
13
14
15
class SquareResponse(BaseModel):
16
    result: int
17
18
19
class CalculatorServer(SimpleResourcesServer):
20
    def setup_webserver(self) -> FastAPI:
21
        app = super().setup_webserver()
22
        app.post("/square")(self.square)
23
        return app
24
25
    async def square(self, body: SquareRequest) -> SquareResponse:
26
        return SquareResponse(result=body.value**2)
27
28
    async def verify(self, body: BaseVerifyRequest) -> BaseVerifyResponse:
29
        used_square = any(
30
            item.type == "function_call" and item.name == "square"
31
            for item in body.response.output
32
        )
33
        return BaseVerifyResponse(
34
            responses_create_params=body.responses_create_params,
35
            response=body.response,
36
            reward=float(used_square),
37
        )
38
39
40
if __name__ == "__main__":
41
    CalculatorServer.run_webserver()

Choose the seed/verify protocol when:

• The Agent harness should own model calls, tool dispatch, retries, or other orchestration.
• Resources Server routes should be exposed directly as model tools.
• One final reward should be computed from the completed trajectory or final environment state.
• You are pairing the environment with simple_agent or a custom Agent Server that implements /seed_session, an Agent harness, and /verify.

Gymnasium Protocol: reset() and step()

The Gymnasium protocol models the environment as a transition function. The Agent Server sends every model response to the Resources Server:

Client → Agent Server: POST /run
Agent Server run() → Resources Server: POST /reset
Agent Server run() → Model Server: POST /v1/responses
Agent Server run() → Resources Server: POST /step
... repeat model → step until terminated or truncated
Agent Server → Client: trajectory and accumulated reward

gymnasium_agent never calls /verify. GymnasiumServer does not register a /verify endpoint; /step provides both transitions and rewards.

Transition API

Subclass GymnasiumServer and implement step(). Override reset() when the environment needs initialization or an initial observation.

1
async def reset(
2
    self,
3
    metadata: dict,
4
    session_id: str | None = None,
5
) -> tuple[str | None, dict]:
6
    ...
7
8
async def step(
9
    self,
10
    action: NeMoGymResponse,
11
    metadata: dict,
12
    session_id: str | None = None,
13
) -> tuple[str | None, float, bool, bool, dict]:
14
    ...

• action is the complete structured model response for the current step. It can contain assistant messages, reasoning items, or function calls.
• metadata contains extra top-level fields from the input row, such as answer, category, or task settings. It does not include responses_create_params.
• session_id identifies the rollout and keys per-episode state. Stateless environments can ignore it.

reset() returns (observation, info). A non-None observation is appended as a user message before the first model call.

step() returns (observation, reward, terminated, truncated, info):

• observation: the next user message, or None when no message is needed.
• reward: reward for this step. gymnasium_agent sums rewards across the episode.
• terminated: True when the episode reaches a natural terminal state.
• truncated: True when a limit or timeout cuts the episode short.
• info: diagnostic metadata and optional tool_outputs for the Agent Server.

Tool Calls as Gymnasium Actions

gymnasium_agent does not dispatch a function call to POST /{name}. Instead:

1. The model returns a NeMoGymResponse.
2. The Agent Server sends that response to /step as action.
3. step() interprets text or structured function calls and executes the corresponding environment operation.
4. For function calls, step() returns results in info["tool_outputs"].
5. The Agent Server converts those results into function_call_output items for the next model call.

1
import json
2
3
from resources_servers.gymnasium import GymnasiumServer
4
5
6
class CalculatorEnv(GymnasiumServer):
7
    async def step(self, action, metadata, session_id=None):
8
        calls = [item for item in action.output if item.type == "function_call"]
9
10
        if calls:
11
            outputs = []
12
            for call in calls:
13
                args = json.loads(call.arguments)
14
                if call.name == "square":
15
                    result = {"result": args["value"] ** 2}
16
                else:
17
                    result = {"error": f"Unknown action: {call.name}"}
18
                outputs.append(self.tool_output(call, result))
19
20
            return None, 0.0, False, False, {"tool_outputs": outputs}
21
22
        return None, 1.0, True, False, {}

Tool schemas still come from responses_create_params.tools, but they describe the actions the model can emit. The Resources Server does not need matching HTTP tool routes because step() performs the dispatch.

Choose the Gymnasium protocol when:

• The Resources Server must process every model response before the episode continues.
• The environment decides the next observation and stopping condition.
• Rewards are naturally assigned per transition.
• The model response itself is the environment action.
• You are pairing the environment with gymnasium_agent or a custom Agent Server that implements the /reset and /step loop.

State Management

Both episode protocols use the Resources Server’s session middleware. It assigns a unique session_id cookie, and the Agent Server preserves that cookie across Resources Server calls in the same rollout.

Store rollout-specific state under the session ID so concurrent episodes remain isolated. State can range from:

• Stateless: grade an answer directly from the response and task metadata.
• Lightweight: store a counter, game board, or conversation state in memory.
• Heavyweight: associate each session with a browser, database, or Docker container.

Seed/Verify State

SimpleResourcesServer does not define a state container. Add one and read the session ID from the FastAPI request:

1
from typing import Any
2
3
from fastapi import Request
4
from pydantic import Field
5
6
from nemo_gym.server_utils import SESSION_ID_KEY
7
8
9
class StatefulServer(SimpleResourcesServer):
10
    session_state: dict[str, Any] = Field(default_factory=dict)
11
12
    async def seed_session(self, request: Request, body: SeedRequest):
13
        session_id = request.session[SESSION_ID_KEY]
14
        self.session_state[session_id] = initialize(body)
15
        return BaseSeedSessionResponse()
16
17
    async def verify(self, request: Request, body: VerifyRequest):
18
        session_id = request.session[SESSION_ID_KEY]
19
        try:
20
            reward = grade(self.session_state[session_id], body)
21
            return make_verify_response(body, reward)
22
        finally:
23
            self.session_state.pop(session_id, None)

Tool endpoints can read the same request.session[SESSION_ID_KEY], so /seed_session, tool calls, and /verify operate on the same environment instance.

Gymnasium State

GymnasiumServer provides self.session_state and passes the session ID to reset() and step():

1
class StatefulEnv(GymnasiumServer):
2
    async def reset(self, metadata, session_id=None):
3
        self.session_state[session_id] = initialize(metadata)
4
        return "Environment ready.", {}
5
6
    async def step(self, action, metadata, session_id=None):
7
        state = self.session_state[session_id]
8
        reward, done = apply_action_and_score(state, action)
9
        return None, reward, done, False, {}

Do not put rollout-specific mutable data in an unkeyed server attribute.

Session Cleanup

• Seed/verify protocol: remove custom state in verify(), preferably in a finally block. SimpleResourcesServer does not automatically clear custom state.
• Gymnasium protocol: when step() returns terminated=True or truncated=True, GymnasiumServer calls close_session() and removes the entry from self.session_state.
• Heavyweight state: override close_session(), release the external resource, and call await super().close_session(session_id).

1
async def close_session(self, session_id):
2
    container = self.session_state.get(session_id, {}).get("container")
3
    if container is not None:
4
        await container.stop()
5
    await super().close_session(session_id)

Ensure a Gymnasium environment returns a terminal or truncated result before the Agent Server’s max_steps is exhausted. Also use timeouts or stale-session reclamation so client failures cannot leak external resources indefinitely.

YAML Configuration

The Resources Server definition has the same YAML shape for both episode protocols. The paired Agent Server determines which protocol is used.

1
calculator:
2
  resources_servers:
3
    calculator:
4
      entrypoint: app.py
5
      domain: knowledge
6
7
calculator_agent:
8
  responses_api_agents:
9
    simple_agent:
10
      entrypoint: app.py
11
      resources_server:
12
        type: resources_servers
13
        name: calculator
14
      model_server:
15
        type: responses_api_models
16
        name: policy_model
17
      datasets:
18
      - name: example
19
        type: example
20
        jsonl_fpath: resources_servers/calculator/data/example.jsonl

The Agent Server’s resources_server.name must match the top-level Resources Server instance name.

Next Steps

• Build a seed/verify environment in Single-Step Environment.
• Add per-rollout state in Stateful Environment.
• Expose Resources Server tools through MCP Resources Server.
• Review common reward designs in Verification Patterns.
• See a multi-step Gymnasium implementation in resources_servers/blackjack.