> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# Browse Environments

> Browse built-in benchmark and training environments.

NeMo Gym includes 100+ environments covering math, coding, reasoning, knowledge, agentic tool use, instruction following, and safety. An environment includes a dataset, agent harness, verifier, and state. Some resources servers include agent composition and datasets and are runnable environments themselves; others are reusable scoring, tool, or state components and are not separate environment entries. Environments marked ✓ have a corresponding benchmark config with a canonical evaluation split and `prepare_script`.

Use the CLI to discover what's available before running:

```bash
gym list benchmarks          # all benchmarks with domain and description
gym list benchmarks aime24   # inspect one benchmark, including its agent harness
gym list environments        # unified environment and benchmark catalog
gym search math              # search environments and benchmarks by capability area
gym list benchmarks --json   # machine-readable output for scripting or CI
```

## Math & Science

| Environment                                                                                                             | Description                                                              | Verification                         | Benchmark |
| ----------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ | ------------------------------------ | --------- |
| [math\_with\_judge](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/math_with_judge)                     | OpenMathReasoning, DAPO, and MathStackOverflow datasets                  | math-verify + LLM judge              | ✓         |
| [math\_with\_autograder](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/math_with_autograder)           | Hard math benchmarks (e.g. IMO AnswerBench)                              | math-verify + LLM autograder         | ✓         |
| [polymath](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/polymath)                                     | Multilingual math across 18 languages and 4 difficulty tiers             | LLM judge (weighted)                 | ✓         |
| [imo\_gradingbench](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/imo_gradingbench)                    | Four-class IMO proof grading                                             | Last-word extraction                 | ✓         |
| [imo\_proofbench\_judge](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/imo_proofbench_judge)           | IMO ProofBench with 0–7 rubric                                           | LLM judge                            | ✓         |
| [math\_proof\_judgement](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/math_proof_judgement)           | Binary proof judgement — model reads a problem and proof, outputs Yes/No | LLM judge                            | ✓         |
| [math\_formal\_lean](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/math_formal_lean)                   | Lean4 formal proof verification                                          | Lean4 compiler                       | ✓         |
| [physics\_judge](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/physics_judge)                          | Open-ended physics QA                                                    | LLM judge + math-verify              | ✓         |
| [ugphysics\_judge](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/ugphysics_judge)                      | Undergraduate physics benchmarks                                         | LLM judge (TRUE/FALSE) + math-verify | ✓         |
| [critpt](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/critpt)                                         | Research-level physics problems scored by the Artificial Analysis API    | LLM judge (external)                 | ✓         |
| [ether0](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/ether0)                                         | Chemistry benchmark verifiers (ether0)                                   | Rule-based + LLM judge               | ✓         |
| [bunsenbench\_chemistry\_mcq](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/bunsenbench_chemistry_mcq) | BunsenBench chemistry multiple-choice benchmark                          | Exact match                          | ✓         |
| [frontierscience\_judge](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/frontierscience_judge)          | FrontierScience olympiad and research answer grading                     | LLM judge                            | ✓         |
| [proof\_verification](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/proof_verification)                | Proof scoring against ground truth                                       | LLM judge + meta-verifier            |           |
| [newton\_bench](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/newton_bench)                            | Scientific law discovery across 12 physics domains                       | Execution                            |           |
| [math\_with\_code](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/math_with_code)                       | Competitive math with calculator tools                                   | Boxed answer + numeric match         |           |

## Coding

| Environment                                                                                                                     | Description                                              | Verification                | Benchmark |
| ------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------- | --------------------------- | --------- |
| [bigcodebench](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/bigcodebench)                                     | BigCodeBench Python solutions against unittest suite     | Code execution              | ✓         |
| [evalplus](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/evalplus)                                             | HumanEval+ and MBPP+ function completion                 | Code execution              | ✓         |
| [code\_gen](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/code_gen)                                            | Competitive coding problem solving                       | Code execution              | ✓         |
| [competitive\_coding\_challenges](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/competitive_coding_challenges) | Contest-style programming problems                       | Code execution              | ✓         |
| [code\_fim](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/code_fim)                                            | Code fill-in-the-middle (HumanEval-Infilling)            | Code execution              | ✓         |
| [bird\_sql](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/bird_sql)                                            | Text-to-SQL on BIRD dev (1,534 SQLite tasks)             | SQL result-set equality     | ✓         |
| [spider2\_lite](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/spider2_lite)                                    | Text-to-SQL on Spider 2.0-Lite (135 enterprise tasks)    | SQL result-set equality     | ✓         |
| [text\_to\_sql](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/text_to_sql)                                     | Text-to-SQL across multiple SQL dialects                 | LLM judge (SQL equivalence) | ✓         |
| [swerl\_gen](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/swerl_gen)                                          | SWE patch and test generation in a sandboxed environment | Code execution (pytest)     | ✓         |
| [scicode](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/scicode)                                               | Multi-step scientific code generation                    | Code execution              | ✓         |
| [cvdp](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/cvdp)                                                     | RTL hardware design code generation                      | Code execution (simulation) | ✓         |
| [openenv](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/openenv)                                               | Python code execution environment via OpenEnv            | Execution (stdout/stderr)   |           |

## Knowledge & Reasoning

| Environment                                                                                                     | Description                                                                                                               | Verification                              | Benchmark |
| --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------- | --------- |
| [gpqa\_diamond](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/gpqa_diamond)                    | Graduate-level science multiple choice (GPQA Diamond)                                                                     | Exact match                               | ✓         |
| [mcqa](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/mcqa)                                     | Multiple-choice QA covering MMLU, GPQA, HLE, LongBench                                                                    | Exact match                               | ✓         |
| [equivalence\_llm\_judge](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/equivalence_llm_judge) | Short-answer QA with LLM-as-a-judge equivalence scoring                                                                   | LLM judge                                 | ✓         |
| [hotpotqa\_qa](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/hotpotqa_qa)                      | Closed-book multi-hop QA (HotPotQA)                                                                                       | SQuAD-style substring match               | ✓         |
| [simpleqa](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/simpleqa)                             | Short-form factual QA with abstention scoring                                                                             | LLM judge (3-tier)                        | ✓         |
| [omniscience](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/omniscience)                       | Factual recall and calibration QA                                                                                         | LLM judge                                 | ✓         |
| [labbench2\_vlm](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/labbench2_vlm)                  | Scientific VLM QA: figures, tables, lab protocols                                                                         | LLM judge                                 | ✓         |
| [litmus\_agent](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/litmus_agent)                    | [Litmus-Bench v0.1](https://huggingface.co/datasets/nvidia/Nemotron-RL-litmus-bench-v0.1) short-answer chemical reasoning | Answer extraction + numeric/boolean match | ✓         |
| [arc\_agi](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/arc_agi)                              | Abstract reasoning puzzles (ARC-AGI)                                                                                      | Exact match (grid)                        | ✓         |
| [nvarc](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/nvarc)                                   | ARC-AGI in inductive (Python) and transductive (grid) modes                                                               | Code execution / exact match              | ✓         |
| [reasoning\_gym](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/reasoning_gym)                  | 100+ tasks: algebra, logic, geometry, graph theory, games                                                                 | Exact match                               |           |
| [multichallenge](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/multichallenge)                 | Multi-turn inference memory and instruction retention                                                                     | LLM judge (rubric)                        |           |
| [mrcr](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/mrcr)                                     | Multi-round coreference resolution                                                                                        | F1 (SequenceMatcher)                      | ✓         |
| [abstention](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/abstention)                         | Train models to abstain when unsure (HotPotQA, three-tier reward)                                                         | LLM judge                                 |           |

## Agentic / Tool Use

| Environment                                                                                                                                                         | Description                                                                              | Verification                   | Benchmark |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- | ------------------------------ | --------- |
| [workplace\_assistant](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/workplace_assistant)                                                          | Workplace tasks: 27 tools, 5 databases, 690 tasks                                        | Rule-based (task completion)   | ✓         |
| [aviary](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/aviary)                                                                                     | Multi-hop QA with Wikipedia search + GSM8k with calculator                               | LLM judge + execution          | ✓         |
| [tavily\_search](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/tavily_search)                                                                      | Web search tool use (Tavily API)                                                         | Execution + optional LLM judge | ✓         |
| [calendar](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/calendar)                                                                                 | Multi-turn calendar scheduling with constraint satisfaction                              | Rule-based (constraints)       | ✓         |
| [finance\_sec\_search](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/finance_sec_search)                                                           | SEC EDGAR filing search for financial analysis                                           | LLM judge + execution          | ✓         |
| [xlam\_fc](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/xlam_fc)                                                                                  | Function calling from Salesforce xlam-60k                                                | Exact match                    | ✓         |
| [single\_step\_tool\_use\_with\_argument\_comparison](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/single_step_tool_use_with_argument_comparison) | Pivot RL for tool use across conversational, SWE, and search domains                     | Argument comparison            | ✓         |
| [tales](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/tales)                                                                                       | Text-adventure games (AlfWorld, ScienceWorld, Jericho, TextWorld) in Gymnasium API style | Rule-based (task completion)   |           |
| [google\_search](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/google_search)                                                                      | MCQA with integrated Google search tool                                                  | Exact match                    |           |

## Instruction Following & Safety

| Environment                                                                                                             | Description                                                                                   | Verification                   | Benchmark |
| ----------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | ------------------------------ | --------- |
| [ifbench](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/ifbench)                                       | IFBench with 57 instruction types (AllenAI library)                                           | LLM judge                      | ✓         |
| [inverse\_if](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/inverse_if)                                | Inverse instruction-following benchmark — instructions that counter conventional expectations | LLM judge (per-task)           | ✓         |
| [structured\_outputs](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/structured_outputs)                | Schema adherence across structured output formats                                             | Rule-based (schema validation) | ✓         |
| [indirect\_prompt\_injection](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/indirect_prompt_injection) | Resistance to injected instructions in tool-use trajectories                                  | Rule-based (attack detection)  | ✓         |
| [jailbreak\_detection](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/jailbreak_detection)              | Jailbreak resistance with Nemotron judge                                                      | LLM judge                      | ✓         |
| [xstest](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/xstest)                                         | Over-refusal calibration (XSTest)                                                             | Rule-based                     | ✓         |
| [instruction\_following](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/instruction_following)          | IFEval and IFBench-style instruction following                                                | LLM judge                      |           |
| [verifif](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/verifif)                                       | VerifIF instruction-following validators (rule-based + LLM judge)                             | Rule-based + LLM judge         |           |
| [format\_verification](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/format_verification)              | Citation format and freeform text formatting                                                  | Regex / rule-based             |           |
| [structeval](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/structeval)                                 | StructEval: JSON, YAML, CSV, TOML, XML schema adherence                                       | Rule-based (schema parsing)    |           |
| [over\_refusal\_detection](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/over_refusal_detection)       | Train models to avoid refusing safe prompts                                                   | LLM judge                      |           |

## Other

| Environment                                                                                        | Description                                                        | Verification              | Benchmark |
| -------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------ | ------------------------- | --------- |
| [asr\_with\_pc](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/asr_with_pc)        | ASR with WER (standard, case-sensitive, punctuation)               | Execution (WER metrics)   | ✓         |
| [wmt\_translation](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/wmt_translation) | Machine translation with BLEU and xCOMET-XXL                       | Execution (COMET metrics) | ✓         |
| [longmt\_eval](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/longmt_eval)         | Document-level translation (SEGALE pipeline + COMETKiwi)           | Execution (neural QE)     | ✓         |
| [vlm\_eval\_kit](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/vlm_eval_kit)      | VLM benchmarks: MMBench, OCRBench, and others                      | Execution (VLMEvalKit)    | ✓         |
| [graphwalks](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/graphwalks)            | Long-context graph BFS/DFS reasoning                               | F1 over node sets         | ✓         |
| [arena\_judge](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/arena_judge)         | Arena Hard v2 pairwise LLM-judge (category-specific, side-swapped) | LLM judge (pairwise)      | ✓         |
| [speed\_bench](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/speed_bench)         | Speculative-decoding throughput measurement                        | vLLM Prometheus metrics   | ✓         |
| [genrm\_compare](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/genrm_compare)     | GenRM pairwise comparison for RLHF training                        | LLM judge (pairwise)      |           |
| [blackjack](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/blackjack)              | Gymnasium-style Blackjack (multi-step)                             | Win/draw/loss             |           |
| [grl\_sokoban](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/grl_sokoban)         | Single-box Sokoban puzzle                                          | Execution (puzzle solved) |           |
| [grl\_tetris](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/grl_tetris)           | Tetris in Gymnasium API style                                      | Rule-based (score/lines)  |           |

#### [Benchmarks](/evaluation/benchmarks)

How to choose a benchmark and interpret results.

#### [Add a Benchmark](/contribute/environments/adding-a-benchmark)

Use the contribution checklist to add a new benchmark.

#### [Build Verifiers](/build-verifiers)

Learn how environments verify agent behavior and compute rewards.