Key Terminology | NeMo Gym

Essential vocabulary for agent evaluation, policy-model training, RL workflows, and NeMo Gym. You’ll encounter these terms throughout the tutorials and documentation.

Deeper treatment: Environments, Evaluation, Training, Architecture.

Overview

A dataset is a JSONL file of tasks (one row = one problem) or a folder of task folders (one folder = one problem). Running the agent on a task produces a rollout (also called a trajectory) — one attempt and its record.

Each run follows:

seed_session — set up isolated state for this attempt
agent loop — call the model (policy), use tools, repeat until done
verify — score the attempt → reward

Two compositions matter:

	Built from	Notes
Agent	Model + Agent harness	The harness owns the loop, tools, context, and stop conditions.
Environment	Dataset + Agent harness + Verifier + State	Everything required for an agent to complete a task. The model is outside the environment.

The agent harness belongs to the agent and is part of the environment definition. In Gym packaging, an environment config names which agent server to run.

The model (also called the policy) is what you call for generation — local weights or a remote API endpoint. In Gym it is usually exposed by the Model server (or your harness can call an endpoint directly). The same environment shape can be used for benchmark eval (fixed taskset / protocol) or for training (rewards or synthetic data); the main differences are task split and contamination controls, not a different environment type.

Concept	In NeMo Gym
Dataset	JSONL rows with `responses_create_params` + `verifier_metadata`, or task folders in the Harbor format — Prepare Tasks
Agent harness	Implemented by the Agent server (`responses_api_agents/`) — Agent Server
Verifier, per-attempt state, env tools	Implemented by the Resources server (`resources_servers/`): `seed_session()` initializes state; tools mutate it; `verify()` scores the attempt — Build Verifiers. For task folders, the `harbor` Environment Server runs the task’s own verifier in its sandbox, or in a fresh one for separate verification — Harbor Environment Server
Model	Served by the Model server (`responses_api_models/`) — wraps local or remote endpoints — Model Server
Protocol	Responses API (Chat Completions converted via middleware when needed)

Glossary

Tasks & rollouts

Term	Definition
Task	One problem for the agent to solve — one dataset row (`responses_create_params` + `verifier_metadata`) or one task folder.
Dataset	Collection of tasks plus metadata needed for scoring (usually JSONL, or a folder of task folders). See Prepare Tasks.
Task folder	One task in the Harbor format: `task.toml` and usually `instruction.md`, `environment/`, `tests/` and `solution/`. It carries its own verifier, so no Resources Server takes part. See Author Task Folders.
Rollout / trajectory	Rollout (verb): execute an agent in an environment — take actions and record what happens. Rollout (noun) / trajectory: the ordered record of one attempt (states, actions, rewards). In Gym these names are aliases; architecture calls the resulting trajectory a rollout. Multiple rollouts per task support metrics like pass@k. See harness conformance results.
Task attempt	One rollout for a specific task. Multiple attempts per task capture different approaches and support pass@k.
Trace	Debug-oriented log of a rollout (timing, tool I/O, metadata) beyond the scored trajectory.
Rollout batch	Multiple rollouts generated together (across tasks for throughput, or grouped on one task for methods like GRPO).
Rollout collection	Running inference, tools, and verification at scale to produce scored rollouts. Start with the Quickstart or Evaluate.

Environment

Term	Definition
Environment	Everything required for an agent to complete a task, excluding the model: `Dataset + Agent Harness + Verifier + State`. See Environments and Build Environments.
Environment Server	Receives each episode and either orders it (seed, agent turn, verification and cleanup) or relays it to an agent that runs it, as `legacy_agent` does. See Design Agent-Environment Interactions.
State	Per-attempt mutable world (files, DB, tool results, and so on). `seed_session()` starts a clean session; tools update that session; it is not a snapshot of the whole environment config.
Verifier	Scores a task attempt into a reward (typically 0–1) via `verify()` on the resources server, or by a task folder’s `tests/` (or its dataset’s shared `verifier.py`), run in the task’s sandbox or a fresh one. Also called scorer or grader. See Build Verifiers and Task Folder Verifiers.
Reward	Numerical score (typically 0.0–1.0) for how well the attempt did — used as eval metrics and as the RL learning signal.
Sandbox	Runtime with isolated execution per attempt (e.g. one container). Broader runtimes (local process, Docker, Apptainer) host execution. See Sandboxes.

Sessions

Resources servers and agent servers keep per-episode state in sessions. Use these names in code, docstrings, error messages, tests, and docs, and do not add synonyms. See Resources Server APIs and Add or Update a Harness.

Term	Definition	Not
Resources session / agent session	The per-episode state a resources server or an agent server holds, named by `resources_session_id` or `agent_session_id`.	Resources Server session
Episode session / episode seed	A session an environment server seeds through `/seed_session` or `/v1/agent_sessions`, and that seed.	typed session, typed seed, Environment Server session
Agent-run seed	The seed an agent’s own `/run` sends to a resources server.	untyped seed
Lock	Makes a session’s seed and close run one at a time: `ResourcesSessionLocks.locked()`.	fence (for the lock)
Fence	The record a close leaves, which refuses a seed that arrives after the close: `ResourcesSessionFences` with `fence()`, `get()` and `raise_if_fenced()`.	closed record, close receipt, tombstone, “remembers the close”
Close retry window	How long a fence lasts: `session_close_retry_window_seconds`.	close window, retry window, retention window
Expire	A timed lifetime runs out: a fence, a failed seed’s record, or an open session (`session_expired`).	forgotten
Saved reply	The first close’s response, which an agent server returns to a retried close.	close receipt, retained close response
Release	Give up what a session or an episode holds, such as its sandbox and in-memory state. A worker also releases its ownership of a session ID, the opposite of acquiring it.	free
Lost	The server no longer holds the session: a 404 on a tool call, or a masked verify with `session_lost`. An agent seed or close whose cookie names a lost agent session is refused with 409: the session is no longer available.	—
Owner	With several workers, the worker that holds a session’s state and serves its requests; its owner file names it.	claim, claimant
Claim	Checkpoint restore only: a replacement attempt takes the restored record; one nobody took is unclaimed.	claim for routing ownership
Terminal	A refusal the environment server does not retry.	final
Conflict	A 409 refusal of a session request. The shared session code (`ResourcesSessionFences` and `SimpleResponsesAPIAgent`) raises `ResourcesSessionConflictError` or `AgentSessionConflictError`, and new session code should too; many existing servers and agents still raise a plain 409 `HTTPException`.	`HTTPException(409, ...)` in new session code

Other features use some of these words in their own senses and keep them:

Receipt: the process supervisor’s termination and cleanup receipts, export receipts, capture receipts, and agentic_if’s plugin and hook receipts.
Tombstone: the tombstones of gdpval’s journal and genrm_compare’s cohorts.
Fence: token capture fences a retired rollout until it is deleted, and the process supervisor fences a delayed harness launch.
Claim: MCP token claims (MCPTokenClaims) and the supervisor’s launch.claim file.
Terminal: the checkpoint stack’s terminal step.
Expire: SandboxSession.expire() ends a harness as if its time limit had passed.

Agent & model

Term	Definition
Model / policy model	The model being trained or evaluated — local weights or a remote endpoint, usually via the Model server. See Model Server.
Agent harness	How the model interacts with the environment: loops model calls, routes tools, manages context, and decides when the task is done. See Agent Server.
Agent	Model + agent harness.
Rollout driver	Coordinates an episode and records its trajectory.
Multi-turn	Dialogue across turns where conversation context (and often state) persists.
Multi-step	Sequential tool calls or intermediate steps in the agent loop before completion (may be single- or multi-turn). See Multi-Step Environment.
Tool use	Function calling — invoking external capabilities (APIs, code execution, databases, and so on).

Evaluation

Term	Definition
Evaluation	Run an agent on tasks, score results, and measure performance. See Evaluation and the Evaluate workflow.
Benchmark	Repeatable, versioned evaluation built on an environment: canonical dataset split, prompt configuration, metrics, and comparison protocol. Not every environment is used as a benchmark. See Benchmarks.
pass@k	Fraction of tasks with at least one success among k rollouts. See Aggregate Metrics.

Training

Term	Definition
SFT (Supervised Fine-Tuning)	Train from examples of good behavior (demonstration data: successful / high-reward rollouts).
RL (Reinforcement Learning)	Improve the policy through environment interaction and reward signals.
Online / offline	Online: update the policy from rewards while interacting (e.g. GRPO). Offline: train from pre-collected rollouts (e.g. SFT, DPO). See Offline training with rollouts.
DPO (Direct Preference Optimization)	Offline preference training from pairs of rollouts (preferred vs dispreferred).
GRPO (Group Relative Policy Optimization)	Online RL that compares groups of rollouts on the same task relative to each other. See Training Tutorials (e.g. NeMo RL GRPO).
Demonstration data	SFT examples from successful (high-reward) rollouts.
Preference pairs	DPO data: same task, high-reward vs low-reward rollouts (preferred vs dispreferred).

Responses API and Chat Completions are listed in the mapping table above; see Architecture and Model Server.