> This page is for version Main (default).
> For other versions, use one of these documentation indexes:
> - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt
> - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt
> - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt
> - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt
> - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt
> - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt
> - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt
> - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# Custom Harness

> Run your own script instead of gym eval run for a benchmark with its own harness.

Some benchmarks do not run through Gym's rollout collection: the harness may provision external machines and drive Gym from there, for example. For those, set `command` on the benchmark. The driver runs your script **instead of** `gym eval run`, in the same container and with the same services running.

```yaml
driver:
  policy_model: policy
  gym_install:
    ref: main
  mounts:
    - /lustre/my-project/harness:/harness:ro
  benchmarks:
    my-harness:
      prepare:
        split: benchmark
      command: |
        python /harness/run_my_harness.py \
          --base-url "$NEMO_GYM_POLICY_BASE_URL" \
          --model "$NEMO_GYM_POLICY_MODEL_NAME" \
          --api-key "$NEMO_GYM_POLICY_API_KEY" \
          --output "$NEMO_GYM_BENCH_DIR/artifacts/results.json"
```

## What your script gets

| Variable                     | Value                                                                                                                                              |
| ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| `NEMO_GYM_BENCH_DIR`         | The benchmark's directory in the run directory, an absolute path. Write results under `$NEMO_GYM_BENCH_DIR/artifacts` so they sit beside the logs. |
| `NEMO_GYM_POLICY_BASE_URL`   | The policy's OpenAI-compatible base URL. Set when `driver.policy_model` is set.                                                                    |
| `NEMO_GYM_POLICY_MODEL_NAME` | The served model name. Set when `driver.policy_model` is set.                                                                                      |
| `NEMO_GYM_POLICY_API_KEY`    | The API key to present. Set when `driver.policy_model` is set.                                                                                     |

The driver clones Gym into a fresh temporary checkout when `gym_install` is set, so a script that sits next to your submit config is not copied there. Put the harness on a shared cluster path, mount it with `driver.mounts`, and call it by absolute path, as `/harness/run_my_harness.py` above.

The variables from `driver.env` are present too, and `driver.container` and `driver.mounts` apply as usual.

## What changes compared with `gym eval run`

* The `command` is emitted verbatim as the last statement of the driver's shell, under `set -euo pipefail`. A multi-line script is fine, and its exit status is the job's.
* If `gym_install` is set, the install step still runs first, and so does `prepare`. Your script starts in the Gym checkout.
* Nothing from `run` is applied, which is why `run` and `command` cannot be combined: `run` only configures `gym eval run`. Put those settings in your script.
* `--model-type`, `require_complete`, observability capture, and the default `rollouts.jsonl` output are properties of `gym eval run`. They do not apply; your script decides where results go and what counts as complete.
* `resumable` cannot be combined with `command`. A resumed job only passes `resume_from_cache` to `gym eval run`, so a custom command would restart from zero.