> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/v25.09/nemo-curator/nemo_curator/eval/llm_judge/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/curator/_mcp/server.

# nemo_curator.eval.llm_judge

## Submodules

* **[`nemo_curator.eval.llm_judge.workflow`](/nemo/curator/nemo-curator/nemo_curator/eval/llm_judge/workflow)**

## Package Contents

### Classes

| Name                                                                         | Description                                  |
| ---------------------------------------------------------------------------- | -------------------------------------------- |
| [`LLMJudgeWorkflow`](#nemo_curator-eval-llm_judge-workflow-LLMJudgeWorkflow) | End-to-end config-driven LLM judge workflow. |

### API

```python
class nemo_curator.eval.llm_judge.LLMJudgeWorkflow(
    judge_config: str | pathlib.Path,
    input_path: str,
    output_path: str,
    input_format: nemo_curator.eval.llm_judge.workflow.DataFormat = 'jsonl',
    output_format: nemo_curator.eval.llm_judge.workflow.DataFormat = 'jsonl',
    files_per_partition: int | None = None,
    language: str | None = None,
    fasttext_langid_model_path: str | None = None,
    min_langid_score: float = 0.3,
    language_text_field: str = 'raw_text',
    checkpoint_path: str | None = None
)
```

Dataclass

**Bases:** [WorkflowBase](/nemo/curator/nemo-curator/nemo_curator/pipeline/workflow#nemo_curator-pipeline-workflow-WorkflowBase)

End-to-end config-driven LLM judge workflow.

Loads a judge config YAML (models, Jinja prompt templates, score rubrics,
and `execution.stages`), starts a Dynamo/vLLM inference server hosting
the configured judge models, then runs one Curator pipeline containing:
reader -> optional FastText language gate -> one NDD `DataDesignerStage`
(+ its filters) per judge stage -> writer.

**`checkpoint_path`** `str | None = None`

---

**`config`** `dict[str, object] = field(init=False)`

---

**`config_path`** `Path = field(init=False)`

---

**`fasttext_langid_model_path`** `str | None = None`

---

**`files_per_partition`** `int | None = None`

---

**`input_format`** `DataFormat = 'jsonl'`

---

**`input_path`** `str`

---

**`judge_config`** `str | Path`

---

**`language`** `str | None = None`

---

**`language_text_field`** `str = 'raw_text'`

---

**`min_langid_score`** `float = 0.3`

---

**`output_format`** `DataFormat = 'jsonl'`

---

**`output_path`** `str`

---

```python
nemo_curator.eval.llm_judge.LLMJudgeWorkflow.__post_init__() -> None
```

```python
nemo_curator.eval.llm_judge.LLMJudgeWorkflow._build_judge_stages(
    endpoint: str
) -> list[tuple[str, data_designer.config.DataDesignerConfigBuilder, list[data_designer.config.ModelProvider], dict[str, object] | None, int | None, list[dict[str, object]]]]
```

```python
nemo_curator.eval.llm_judge.LLMJudgeWorkflow.run() -> nemo_curator.pipeline.workflow.WorkflowRunResult
```

Run the complete LLM judge pipeline.

**Returns:** [`WorkflowRunResult`](/nemo/curator/nemo-curator/nemo_curator/pipeline/workflow#nemo_curator-pipeline-workflow-WorkflowRunResult)

WorkflowRunResult containing the pipeline output tasks and timing metadata.

## Docs

- [nemo_curator.eval.llm_judge.workflow](https://docs.nvidia.com/nemo/curator/v25.09/nemo-curator/nemo_curator/eval/llm_judge/workflow.md)