> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/evaluator/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/evaluator/_mcp/server.

# Installation

## Requirements

* Python 3.12 or 3.13
* An OpenAI-compatible model endpoint (NVIDIA API Catalog, vLLM, NIM, etc.)

## Install from Source

```bash
git clone https://github.com/NVIDIA-NeMo/Evaluator.git
cd Evaluator
pip install -e ".[all]"
```

## Install Extras

| Extra       | Command                         | What it adds                                                                          |
| ----------- | ------------------------------- | ------------------------------------------------------------------------------------- |
| `scoring`   | `pip install -e ".[scoring]"`   | sympy for symbolic math comparison                                                    |
| `stats`     | `pip install -e ".[stats]"`     | scipy for confidence intervals, McNemar significance testing, and regression analysis |
| `skills`    | `pip install -e ".[skills]"`    | NeMo Skills benchmark integration                                                     |
| `harbor`    | `pip install -e ".[harbor]"`    | Harbor agent integration (OpenHands, Terminus-2, etc.)                                |
| `lm-eval`   | `pip install -e ".[lm-eval]"`   | LM-Evaluation-Harness integration                                                     |
| `inspect`   | `pip install -e ".[inspect]"`   | Inspect AI log export (`inspect_ai`-compatible `EvalLog` files)                       |
| `ray`       | `pip install -e ".[ray]"`       | Ray distributed launcher                                                              |
| `harnesses` | `pip install -e ".[harnesses]"` | lm-evaluation-harness tasks                                                           |
| `export`    | `pip install -e ".[export]"`    | WandB and MLflow experiment tracker export                                            |
| `docs`      | `pip install -e ".[docs]"`      | Sphinx, NVIDIA theme, mermaid for building docs                                       |
| `all`       | `pip install -e ".[all]"`       | Common runtime integrations: scoring, stats, ray, lm-eval, and Harbor                 |
| `dev`       | `pip install -e ".[dev]"`       | pytest, ruff, and common development extras                                           |

## Verify Installation

```bash
nel --version
nel list
```

Expected output:

```
nemo-evaluator 0.12.0

Built-in benchmarks:
  drop                           nel eval run --bench drop
  gpqa                           nel eval run --bench gpqa
  gsm8k                          nel eval run --bench gsm8k
  ...
```

## Docker

```bash
docker build -t nemo-evaluator .
docker run nemo-evaluator nel list
```

## Next Steps

Proceed to the [Quickstart](/get-started/quickstart) to run your first evaluation.