> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/curator/_mcp/server.

# nemo_curator.stages.text.download.arxiv.stage

## Module Contents

### Classes

| Name                                                                                                    | Description                                                |
| ------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- |
| [`ArxivDownloadExtractStage`](#nemo_curator-stages-text-download-arxiv-stage-ArxivDownloadExtractStage) | Composite stage for downloading and processing Arxiv data. |

### API

```python
class nemo_curator.stages.text.download.arxiv.stage.ArxivDownloadExtractStage(
    download_dir: str = './arxiv_downloads',
    url_limit: int | None = None,
    record_limit: int | None = None,
    add_filename_column: bool | str = True,
    log_frequency: int = 1000,
    verbose: bool = False
)
```

**Bases:** `DocumentDownloadExtractStage`

Composite stage for downloading and processing Arxiv data.

This pipeline:

1. Generates Arxiv dump URLs
2. Downloads Arxiv .tar files
3. Extracts articles from the tar files
4. Cleans and extracts text from LaTeX files

**`downloader`**

---

**`extractor`** `= ArxivExtractor()`

---

**`iterator`** `= ArxivIterator(log_frequency=log_frequency)`

---

**`name`** `= 'arxiv_pipeline'`

---

**`url_generator`** `= ArxivUrlGenerator()`

---

```python
nemo_curator.stages.text.download.arxiv.stage.ArxivDownloadExtractStage.decompose() -> list[nemo_curator.stages.base.ProcessingStage]
```

Decompose this composite stage into its constituent stages.

```python
nemo_curator.stages.text.download.arxiv.stage.ArxivDownloadExtractStage.get_description() -> str
```

Get a description of this composite stage.