> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/curator/_mcp/server.

# nemo_curator.stages.text.download.common_crawl.extract

## Module Contents

### Classes

| Name                                                                                                           | Description |
| -------------------------------------------------------------------------------------------------------------- | ----------- |
| [`CommonCrawlHTMLExtractor`](#nemo_curator-stages-text-download-common_crawl-extract-CommonCrawlHTMLExtractor) | -           |

### API

```python
class nemo_curator.stages.text.download.common_crawl.extract.CommonCrawlHTMLExtractor(
    algorithm: nemo_curator.stages.text.download.html_extractors.HTMLExtractorAlgorithm | str | None = None,
    algorithm_kwargs: dict | None = None,
    stop_lists: dict[str, frozenset[str]] | None = None
)
```

**Bases:** `DocumentExtractor`

```python
nemo_curator.stages.text.download.common_crawl.extract.CommonCrawlHTMLExtractor.extract(
    record: dict[str, typing.Any]
) -> dict[str, typing.Any] | None
```

Extract text from HTML content in the record.

Takes a record dict containing "content" field with HTML and returns
a new dict with only the output columns: url, warc\_id, source\_id, language, text.

```python
nemo_curator.stages.text.download.common_crawl.extract.CommonCrawlHTMLExtractor.input_columns() -> list[str]
```

```python
nemo_curator.stages.text.download.common_crawl.extract.CommonCrawlHTMLExtractor.output_columns() -> list[str]
```