> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/curator/_mcp/server.

# nemo_curator.stages.text.filters.doc_filter

## Module Contents

### Classes

| Name                                                                            | Description                                             |
| ------------------------------------------------------------------------------- | ------------------------------------------------------- |
| [`DocumentFilter`](#nemo_curator-stages-text-filters-doc_filter-DocumentFilter) | An abstract base class for text-based document filters. |

### API

```python
class nemo_curator.stages.text.filters.doc_filter.DocumentFilter()
```

Abstract

An abstract base class for text-based document filters.

This class serves as a template for creating specific document filters
in the library. Subclasses should implement the abstract methods to
define custom filtering behavior.

**`_name`** `= self.__class__.__name__`

---

**`name`** `str`

---

**`ngrams`** `dict`

---

**`paragraphs`** `list`

---

**`sentences`** `list`

---

```python
nemo_curator.stages.text.filters.doc_filter.DocumentFilter.keep_document(
    scores: float | list[int | float]
) -> bool
```

abstract

Determine whether to keep a document based on its scores.

This method should be implemented by subclasses to define the
criteria for keeping or discarding a document based on the
scores calculated by score\_document().

**Parameters:**

**`scores`** `float | list[int | float]`

The score or set of scores returned by score\_document().
The type should match what is returned by score\_document().

---

**Returns:** `bool`

True if the document should be kept, False otherwise.

**Raises:**

* `NotImplementedError`: If the method is not implemented in a subclass.

```python
nemo_curator.stages.text.filters.doc_filter.DocumentFilter.score_document(
    text: str
) -> float | list[int | float]
```

abstract

Calculate a score for the given document text.

This method should be implemented by subclasses to define how
a document's text is evaluated and scored.

**Parameters:**

**`text`** `str`

The text content of the document to be scored.

---

**Returns:** `float | list[int | float]`

A score or set of scores representing the document's

**Raises:**

* `NotImplementedError`: If the method is not implemented in a subclass.