> This page is for version 0.7.0. > For other versions, use one of these documentation indexes: > - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt > - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt > - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt > - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt > - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt > - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt > - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt > - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server. # Resources Server APIs > Understand the supported Resources Server protocol APIs and choose the appropriate one to build your Environment # Resources Server APIs A Resources Server implements the Environment interface related to: its tools, state, and reward logic. > **Info** > > **Agent Server context** > > The Agent is the behavior that decides what to do next, typically with different components like a Harness and a policy-model. > The Agent Server is the HTTP transport for two internal methods: > > * `run()` is exposed as `/run` and implements part of the episode orchestration associated with the Resources Server seed session, delegation of the Agent loop to responses() and call to the verifier. > * `responses()` is exposed as `/v1/responses` and implements Agent behavior. Depending on the Agent Server, this can be a complete agent harness or one policy-model call. > > In `simple_agent`, `run()` delegates the Agent loop to `responses()` before requesting final verification. In `gymnasium_agent`, `run()` owns the model-and-environment loop and does not invoke `responses()` to the tool calling. ## Resources Server Protocol Landscape NeMo Gym ships two framework-standard base protocols for coordinating an episode between the Agent and the Environment. Agent behavior and rollout orchestration are implemented by `responses()` and `run()` behind the Agent Server's HTTP endpoints; the Environment is implemented by the Resources Server. Each protocol has a matching built-in Agent Server: `simple_agent` for the `/seed_session` and `/verify` interface, and `gymnasium_agent` for the `/reset` and `/step` interface. | Base protocol | Resources Server API | Matching built-in Agent Server | Reward contract | | ------------- | ----------------------------- | ------------------------------ | -------------------------------------------------------- | | Seed/verify | `/seed_session` and `/verify` | `simple_agent` | `/verify` scores the completed trajectory once. | | Gymnasium | `/reset` and `/step` | `gymnasium_agent` | Every `/step` returns a reward; `/run` accumulates them. | Pick one base protocol and build the Resources Server and Agent Server to match. `GymnasiumServer` exposes `/reset` and `/step` and does not expose `/verify`. `SimpleResourcesServer` defines `/seed_session` and `/verify` but can be extended with additional environment-specific routes. > **Note** > > The two base protocols do not describe every concrete Agent–Environment interface in the repository. Some environments extend the seed/verify protocol with custom transition endpoints. Other environments are embedded inside the Agent and do not use a Resources Server during `/run`. ### Seed/Verify Protocol Extensions Several environments keep the `/seed_session` and `/verify` lifecycle while adding bespoke transition APIs. These extensions require a matching Agent Server; sharing an endpoint name such as `/step` does not make their request and response contracts interchangeable. | Environment | Episode flow | Extension contract | | ----------- | --------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | Aviary | `/seed_session` → repeated `/step` → `/close` → `/verify` | `/step` receives an environment ID and a list of function-call actions, then returns an observation, reward, and `done`. | | ToolSandbox | `/seed_session` → repeated `/step` → `/close` → `/verify` | `/step` accepts function calls or a natural-language response for the user simulator. `/close` finalizes environment scoring before `/verify`. | | OpenEnv | `/seed_session` → interaction → `/verify` | Depending on its mode, interaction uses a custom `/step` action or direct per-tool HTTP routes. `/verify` returns the accumulated environment reward. | Some Agent Servers also use variants of the seed/verify sequence—for example, skipping a no-op `/seed_session`, treating it as optional, or calling `/verify` without tool execution. These are Agent Server behaviors built around the seed/verify API rather than new Resources Server base classes. ### Agent-Embedded Environments Some Agent Servers embed the environment or an external benchmark harness in their own process. Examples include OSWorld, Tau2, SWE wrappers, Harbor, PinchBench, and Verifiers. Their `/run` implementation computes reward without calling a Resources Server, so they do not implement a Resources Server episode protocol. ### Action and Tool Transports The episode protocol defines initialization, transitions, and scoring. The action transport separately defines how the Agent affects the Environment during the episode. | Transport | Interaction | | ---------------------- | ---------------------------------------------------------------------------------------------------------------- | | Direct HTTP tools | The model emits a function call, and the Agent Server sends `POST /{tool_name}` to the Resources Server. | | MCP | An MCP-capable harness calls tools through the Resources Server's `/mcp` endpoint. | | Gymnasium step action | The Agent Server sends the complete model response to `/step`; the Resources Server interprets and executes it. | | Bespoke transition API | A matching Agent Server and Resources Server exchange environment-specific `/step`, `/close`, or other requests. | | Agent-embedded tools | The Agent Server or its benchmark harness executes tools internally without a Resources Server. | ## Seed/Verify Protocol: `seed_session()` and `verify()` The seed/verify protocol brackets the Agent harness with environment initialization and final verification: ```text Client → Agent Server: POST /run Agent Server → Resources Server: POST /seed_session Agent Server run() → Agent Server responses(): POST /v1/responses responses() ↔ Model Server responses() → Resources Server tool endpoint ... repeat the model/tool Agent loop Agent Server → Resources Server: POST /verify Agent Server → Client: completed trajectory and reward ``` For the built-in `simple_agent`, `run()` performs the lifecycle orchestration and `responses()` implements the Agent loop. The Agent Server exposes those methods as `/run` and `/v1/responses`, respectively. ### Lifecycle API `SimpleResourcesServer` exposes: ```python async def seed_session( self, body: BaseSeedSessionRequest, ) -> BaseSeedSessionResponse: ... async def verify( self, body: BaseVerifyRequest, ) -> BaseVerifyResponse: ... ``` * **`body` in `seed_session()`** contains fields used to initialize an episode. Define a `BaseSeedSessionRequest` subclass for fields such as an initial board, task ID, or sandbox image. The default implementation is a no-op. * **`body.responses_create_params` in `verify()`** is the original model request. * **`body.response` in `verify()`** is the completed `NeMoGymResponse`, including the trajectory assembled by the Agent Server. * **Custom task metadata** requires a `BaseVerifyRequest` subclass. Declare fields such as `answer` or `expected_state`; undeclared fields are not retained by the base Pydantic model. * **`request: Request`**, when added to either method signature, provides access to the per-rollout session ID. ### Direct HTTP Tool-Calling Protocol The seed/verify episode protocol does not itself define how the Agent acts on the environment. `simple_agent` supplies a direct HTTP function-tool protocol inside `responses()`: 1. The input row declares function schemas in `responses_create_params.tools`. 2. The Agent Server sends those schemas to the Model Server. 3. The model returns a structured `function_call` with `name`, `arguments`, and `call_id`. 4. `simple_agent` parses `arguments` and sends `POST /{name}` to the Resources Server. 5. The Resources Server's response becomes a `function_call_output` associated with the same `call_id`. 6. `responses()` sends the updated trajectory back to the model and repeats until the model returns a final response or the step limit is reached. The tool name in the input schema must match the Resources Server route: ```python class SquareRequest(BaseModel): value: int class SquareResponse(BaseModel): result: int class CalculatorServer(SimpleResourcesServer): def setup_webserver(self) -> FastAPI: app = super().setup_webserver() app.post("/square")(self.square) return app async def square(self, body: SquareRequest) -> SquareResponse: return SquareResponse(result=body.value**2) ``` For a model response such as: ```json { "type": "function_call", "name": "square", "arguments": "{\"value\": 4}", "call_id": "call_123" } ``` `simple_agent` calls `POST /square` with `{"value": 4}`. The Agent Server performs this dispatch; the Resources Server only implements the endpoint. > **Info** > > Resources Server tools can also be exposed over MCP. MCP changes the tool transport, not the episode protocol: the rollout can still use `/seed_session` and `/verify`. See [MCP Resources Server](/environment-tutorials/mcp-resources-server). ### Minimal Seed/Verify Environment The complete server below combines the tool route with final trajectory verification: ```python from fastapi import FastAPI from pydantic import BaseModel from nemo_gym.base_resources_server import ( BaseVerifyRequest, BaseVerifyResponse, SimpleResourcesServer, ) class SquareRequest(BaseModel): value: int class SquareResponse(BaseModel): result: int class CalculatorServer(SimpleResourcesServer): def setup_webserver(self) -> FastAPI: app = super().setup_webserver() app.post("/square")(self.square) return app async def square(self, body: SquareRequest) -> SquareResponse: return SquareResponse(result=body.value**2) async def verify(self, body: BaseVerifyRequest) -> BaseVerifyResponse: used_square = any( item.type == "function_call" and item.name == "square" for item in body.response.output ) return BaseVerifyResponse( responses_create_params=body.responses_create_params, response=body.response, reward=float(used_square), ) if __name__ == "__main__": CalculatorServer.run_webserver() ``` Choose the seed/verify protocol when: * The Agent harness should own model calls, tool dispatch, retries, or other orchestration. * Resources Server routes should be exposed directly as model tools. * One final reward should be computed from the completed trajectory or final environment state. * You are pairing the environment with `simple_agent` or a custom Agent Server that implements `/seed_session`, an Agent harness, and `/verify`. ## Gymnasium Protocol: `reset()` and `step()` The Gymnasium protocol models the environment as a transition function. The Agent Server sends every model response to the Resources Server: ```text Client → Agent Server: POST /run Agent Server run() → Resources Server: POST /reset Agent Server run() → Model Server: POST /v1/responses Agent Server run() → Resources Server: POST /step ... repeat model → step until terminated or truncated Agent Server → Client: trajectory and accumulated reward ``` `gymnasium_agent` never calls `/verify`. `GymnasiumServer` does not register a `/verify` endpoint; `/step` provides both transitions and rewards. ### Transition API Subclass `GymnasiumServer` and implement `step()`. Override `reset()` when the environment needs initialization or an initial observation. ```python async def reset( self, metadata: dict, session_id: str | None = None, ) -> tuple[str | None, dict]: ... async def step( self, action: NeMoGymResponse, metadata: dict, session_id: str | None = None, ) -> tuple[str | None, float, bool, bool, dict]: ... ``` * **`action`** is the complete structured model response for the current step. It can contain assistant messages, reasoning items, or function calls. * **`metadata`** contains extra top-level fields from the input row, such as `answer`, `category`, or task settings. It does not include `responses_create_params`. * **`session_id`** identifies the rollout and keys per-episode state. Stateless environments can ignore it. `reset()` returns `(observation, info)`. A non-`None` observation is appended as a user message before the first model call. `step()` returns `(observation, reward, terminated, truncated, info)`: * **`observation`**: the next user message, or `None` when no message is needed. * **`reward`**: reward for this step. `gymnasium_agent` sums rewards across the episode. * **`terminated`**: `True` when the episode reaches a natural terminal state. * **`truncated`**: `True` when a limit or timeout cuts the episode short. * **`info`**: diagnostic metadata and optional `tool_outputs` for the Agent Server. ### Tool Calls as Gymnasium Actions `gymnasium_agent` does not dispatch a function call to `POST /{name}`. Instead: 1. The model returns a `NeMoGymResponse`. 2. The Agent Server sends that response to `/step` as `action`. 3. `step()` interprets text or structured function calls and executes the corresponding environment operation. 4. For function calls, `step()` returns results in `info["tool_outputs"]`. 5. The Agent Server converts those results into `function_call_output` items for the next model call. ```python import json from resources_servers.gymnasium import GymnasiumServer class CalculatorEnv(GymnasiumServer): async def step(self, action, metadata, session_id=None): calls = [item for item in action.output if item.type == "function_call"] if calls: outputs = [] for call in calls: args = json.loads(call.arguments) if call.name == "square": result = {"result": args["value"] ** 2} else: result = {"error": f"Unknown action: {call.name}"} outputs.append(self.tool_output(call, result)) return None, 0.0, False, False, {"tool_outputs": outputs} return None, 1.0, True, False, {} ``` Tool schemas still come from `responses_create_params.tools`, but they describe the actions the model can emit. The Resources Server does not need matching HTTP tool routes because `step()` performs the dispatch. Choose the Gymnasium protocol when: * The Resources Server must process every model response before the episode continues. * The environment decides the next observation and stopping condition. * Rewards are naturally assigned per transition. * The model response itself is the environment action. * You are pairing the environment with `gymnasium_agent` or a custom Agent Server that implements the `/reset` and `/step` loop. ## State Management Both episode protocols use the Resources Server's session middleware. It assigns a unique `session_id` cookie, and the Agent Server preserves that cookie across Resources Server calls in the same rollout. Store rollout-specific state under the session ID so concurrent episodes remain isolated. State can range from: * **Stateless**: grade an answer directly from the response and task metadata. * **Lightweight**: store a counter, game board, or conversation state in memory. * **Heavyweight**: associate each session with a browser, database, or Docker container. ### Seed/Verify State `SimpleResourcesServer` does not define a state container. Add one and read the session ID from the FastAPI request: ```python from typing import Any from fastapi import Request from pydantic import Field from nemo_gym.server_utils import SESSION_ID_KEY class StatefulServer(SimpleResourcesServer): session_state: dict[str, Any] = Field(default_factory=dict) async def seed_session(self, request: Request, body: SeedRequest): session_id = request.session[SESSION_ID_KEY] self.session_state[session_id] = initialize(body) return BaseSeedSessionResponse() async def verify(self, request: Request, body: VerifyRequest): session_id = request.session[SESSION_ID_KEY] try: reward = grade(self.session_state[session_id], body) return make_verify_response(body, reward) finally: self.session_state.pop(session_id, None) ``` Tool endpoints can read the same `request.session[SESSION_ID_KEY]`, so `/seed_session`, tool calls, and `/verify` operate on the same environment instance. ### Gymnasium State `GymnasiumServer` provides `self.session_state` and passes the session ID to `reset()` and `step()`: ```python class StatefulEnv(GymnasiumServer): async def reset(self, metadata, session_id=None): self.session_state[session_id] = initialize(metadata) return "Environment ready.", {} async def step(self, action, metadata, session_id=None): state = self.session_state[session_id] reward, done = apply_action_and_score(state, action) return None, reward, done, False, {} ``` Do not put rollout-specific mutable data in an unkeyed server attribute. ### Session Cleanup * **Seed/verify protocol**: remove custom state in `verify()`, preferably in a `finally` block. `SimpleResourcesServer` does not automatically clear custom state. * **Gymnasium protocol**: when `step()` returns `terminated=True` or `truncated=True`, `GymnasiumServer` calls `close_session()` and removes the entry from `self.session_state`. * **Heavyweight state**: override `close_session()`, release the external resource, and call `await super().close_session(session_id)`. * **Environment Server sessions**: an Environment Server seeds with a `ResourcesSeedSessionRequest` and closes through `/close_session`. `SimpleResourcesServer` answers both with no-op defaults, so a server without per-rollout state needs no changes. A server with state overrides `seed_session()` and `close_resources_session()`, which is named differently from the Gymnasium `close_session()` helper. ```python async def close_session(self, session_id): container = self.session_state.get(session_id, {}).get("container") if container is not None: await container.stop() await super().close_session(session_id) ``` Ensure a Gymnasium environment returns a terminal or truncated result before the Agent Server's `max_steps` is exhausted. Also use timeouts or stale-session reclamation so client failures cannot leak external resources indefinitely. ## YAML Configuration The Resources Server definition has the same YAML shape for both episode protocols. The paired Agent Server determines which protocol is used. #### Seed/verify ```yaml calculator: resources_servers: calculator: entrypoint: app.py domain: knowledge calculator_agent: responses_api_agents: simple_agent: entrypoint: app.py resources_server: type: resources_servers name: calculator model_server: type: responses_api_models name: policy_model datasets: - name: example type: example jsonl_fpath: resources_servers/calculator/data/example.jsonl ``` #### Gymnasium ```yaml calculator: resources_servers: calculator: entrypoint: app.py domain: knowledge calculator_agent: responses_api_agents: gymnasium_agent: entrypoint: app.py resources_server: type: resources_servers name: calculator model_server: type: responses_api_models name: policy_model max_steps: 10 datasets: - name: example type: example jsonl_fpath: resources_servers/calculator/data/example.jsonl ``` The Agent Server's `resources_server.name` must match the top-level Resources Server instance name. For an Environment Server, add `taskset: ` to the dataset declaration and map that name in `environment_server_routes`. The normal `gym dataset collate` and `gym eval run --benchmark ...` paths then convert prepared flat rows to `task_id` and `task_input`; no separate conversion command is needed. Collation leaves task-owned fields flat inside `task_input`; each Environment Server validates its own input when dispatched. The built-in `single_agent_turn` server also accepts the existing `{responses_create_params, task_data}` shape. Its `task_data` key is reserved as that container; additional flat fields are merged into it, with conflicting values rejected. Dataset metrics count custom rows without requiring Responses API fields; response-specific metrics are collected only when those fields validate. The source JSONL stays unchanged and can still be shared with flat-row recipes. Omit `taskset` to keep the current flat-row format. This option accepts flat input only; already materialized input should be passed directly with `--input`. For example, add these keys to a config whose `swebench_pro_hermes` Environment Server binds the `swebench_pro_hermes_resources_server` instance: ```yaml environment_server_routes: swebench_pro: swebench_pro_hermes swebench_pro_hermes_resources_server: resources_servers: swebench_pro: datasets: - name: swebench_pro type: benchmark taskset: swebench_pro jsonl_fpath: benchmarks/swebench/data/swebench_pro_benchmark.jsonl prepare_script: benchmarks/swebench/pro/prepare.py ``` `environment_routing_mode` does not need to change: converted rows route by their taskset in every mode. They reject run-level `+responses_create_params.*` overrides, `num_repeats_add_seed`, and run-level `skills`. Set response parameters in the source row before collation; benchmark prompt configs still apply to input messages. The task's qualified ID is `/`. The ID is taken from `task_id`, `problem_id`, or `instance_id`, in that order. Without an explicit ID, an existing `_ng_task_index` takes precedence; otherwise collation uses a zero-based source-row index across all datasets of the same type, before repeats. Adding or reordering other datasets in that collation can shift these positional IDs. These tasksets route to an Environment Server, not to an agent, so their datasets do not set `agent`. Benchmark discovery, `gym eval prepare`, and collation check the route instead: every agent the Environment Server references must exist, a dataset declared on an agent must be one of them, and a dataset declared on a resources server must be the one the Environment Server binds. The agent does not need its own `resources_server` binding. An Environment Server can front several agents through fields other than `agent_server`, such as a `user_agent` and an `assistant_agent`. When it does and names no `agent_server`, `gym eval list` shows the Environment Server instead of an agent. Collation preserves existing collector task indices, rollout IDs, and retry attempts outside `task_input`. ## Next Steps * Build a seed/verify environment in [Single-Step Environment](/environment-tutorials/single-step-environment). * Add per-rollout state in [Stateful Environment](/environment-tutorials/stateful-environment). * Expose Resources Server tools through [MCP Resources Server](/environment-tutorials/mcp-resources-server). * Review common reward designs in [Verification Patterns](/build-verifiers/verification-patterns). * See a multi-step Gymnasium implementation in [`resources_servers/blackjack`](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/blackjack).