Skip to navigation

Prepare Tasks

View as MarkdownOpen in Claude

A task is one unit of work that an agent attempts and a verifier scores. NeMo Gym reads tasks in two forms.

Two kinds of task formats

JSONL rows plus a Resources ServerHarbor task folders
Best fitMany tasks that share one verifier and toolset, such as math, instruction following or tool calling.Agent tasks that need their own software environment, such as terminal and coding tasks, and datasets from the Harbor hub.
What you writeOne line per task with the prompt and anything your verifier needs to score it, plus a Resources Server: a Python server with the tools and the scoring code shared by every task. A config file connects them to an agent. See Data Format.One folder per task with the instruction, the container the agent works in, the tests that score it and, optionally, a reference solution. See Task folder layout.
Where verification runsIn the Resources Server’s verify().In the task’s sandbox, or a fresh one for separate verification, where the task’s tests/, or the dataset’s shared verifier.py, writes the reward.

JSONL rows run on an Environment Server, such as single_agent_turn, that drives an Agent Server and calls your Resources Server to seed each task and verify the attempt. See Design Agent-Environment Interactions to choose the Environment Server, and Build Environments to build a Resources Server end to end.

Harbor task folders run on the harbor Environment Server, which starts a sandbox from each folder, runs the agent session in it, then runs the task’s verifier in that sandbox, or in a fresh one for separate verification. No Resources Server takes part. Start with Author Task Folders, and see Harbor Environment Server for how an episode runs.

Where to go next

  • JSONL Datasets: the row format, dataset configuration and validation, then guides to prepare, download and render JSONL datasets.
  • Harbor Task Folders: write, validate and run task folders, with references for the format, verifiers, multi-step and multi-container tasks.