> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/curator/_mcp/server.

# nemo_curator.stages.text.download.wikipedia.extract

## Module Contents

### Classes

| Name                                                                                            | Description                                                |
| ----------------------------------------------------------------------------------------------- | ---------------------------------------------------------- |
| [`WikipediaExtractor`](#nemo_curator-stages-text-download-wikipedia-extract-WikipediaExtractor) | Extractor for Wikipedia articles from MediaWiki XML dumps. |

### Data

[`CAT_ALIASES`](#nemo_curator-stages-text-download-wikipedia-extract-CAT_ALIASES)

[`MEDIA_ALIASES`](#nemo_curator-stages-text-download-wikipedia-extract-MEDIA_ALIASES)

### API

```python
class nemo_curator.stages.text.download.wikipedia.extract.WikipediaExtractor(
    language: str = 'en'
)
```

**Bases:** `DocumentExtractor`

Extractor for Wikipedia articles from MediaWiki XML dumps.

```python
nemo_curator.stages.text.download.wikipedia.extract.WikipediaExtractor._create_filter_functions(
    re_rm_wikilink: re.Pattern[str],
    re_clean_wikilink: re.Pattern[str]
) -> tuple
```

Create filter functions for Wikipedia content processing.

```python
nemo_curator.stages.text.download.wikipedia.extract.WikipediaExtractor._create_filters() -> tuple[re.Pattern[str], re.Pattern[str], re.Pattern[str]]
```

Create regex patterns for filtering Wikipedia content.

```python
nemo_curator.stages.text.download.wikipedia.extract.WikipediaExtractor._process_sections(
    wikicode: typing.Any,
    re_rm_magic: re.Pattern[str],
    rm_wikilink: typing.Any,
    rm_tag: typing.Any,
    is_category: typing.Any,
    try_replace_obj: typing.Any,
    try_remove_obj: typing.Any
) -> str
```

Process sections of the Wikipedia article.

```python
nemo_curator.stages.text.download.wikipedia.extract.WikipediaExtractor.extract(
    record: dict[str, typing.Any]
) -> dict[str, typing.Any] | None
```

Extract and clean Wikipedia article content.

**Parameters:**

**`record`** `dict[str, Any]`

Record containing raw\_content field with Wikipedia markup

---

**Returns:** `dict[str, Any] | None`

Dict with cleaned text and metadata, or None if extraction fails

```python
nemo_curator.stages.text.download.wikipedia.extract.WikipediaExtractor.input_columns() -> list[str]
```

Define the input columns expected by this extractor.

```python
nemo_curator.stages.text.download.wikipedia.extract.WikipediaExtractor.output_columns() -> list[str]
```

Define the output columns produced by this extractor.

```python
nemo_curator.stages.text.download.wikipedia.extract.CAT_ALIASES = {'ab': ['Категория', 'Акатегориа'], 'ace': ['Kawan', 'Kategori'], 'af': ['Katego...
```

```python
nemo_curator.stages.text.download.wikipedia.extract.MEDIA_ALIASES = {'ab': ['Медиа', 'Файл', 'Афаил', 'Амедиа', 'Изображение'], 'ace': ['Beureukaih'...
```