> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/curator/_mcp/server.

# nemo_curator.stages.text.io.reader.parquet

## Module Contents

### Classes

| Name                                                                                   | Description                                                         |
| -------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
| [`ParquetReader`](#nemo_curator-stages-text-io-reader-parquet-ParquetReader)           | Composite stage for reading Parquet files.                          |
| [`ParquetReaderStage`](#nemo_curator-stages-text-io-reader-parquet-ParquetReaderStage) | Stage that processes a group of Parquet files into a DocumentBatch. |

### API

```python
class nemo_curator.stages.text.io.reader.parquet.ParquetReader(
    file_paths: str | list[str],
    files_per_partition: int | None = None,
    blocksize: int | str | None = None,
    fields: list[str] | None = None,
    read_kwargs: dict[str, typing.Any] | None = None,
    file_extensions: list[str] = (lambda: FILETYPE_TO_DEFAUL...,
    task_type: typing.Literal['document', 'image', 'video', 'audio'] = 'document',
    _generate_ids: bool = False,
    _assign_ids: bool = False,
    name: str = 'parquet_reader'
)
```

Dataclass

**Bases:** [CompositeStage\[EmptyTask, DocumentBatch\]](/nemo-curator/nemo_curator/stages/base#nemo_curator-stages-base-CompositeStage)

Composite stage for reading Parquet files.

This high-level stage decomposes into:

1. FilePartitioningStage - partitions files into groups
2. ParquetReaderStage - reads file groups into DocumentBatches

**`_assign_ids`** `bool = False`

---

**`_generate_ids`** `bool = False`

---

**`blocksize`** `int | str | None = None`

---

**`fields`** `list[str] | None = None`

---

**`file_extensions`** `list[str]`

---

**`file_paths`** `str | list[str]`

---

**`files_per_partition`** `int | None = None`

---

**`name`** `str = 'parquet_reader'`

---

**`read_kwargs`** `dict[str, Any] | None = None`

---

**`task_type`** `Literal['document', 'image', 'video', 'audio'] = 'document'`

---

```python
nemo_curator.stages.text.io.reader.parquet.ParquetReader.__post_init__()
```

Initialize parent class after dataclass initialization.

```python
nemo_curator.stages.text.io.reader.parquet.ParquetReader.decompose() -> list[nemo_curator.stages.text.io.reader.parquet.ParquetReaderStage]
```

Decompose into file partitioning and processing stages.

```python
nemo_curator.stages.text.io.reader.parquet.ParquetReader.get_description() -> str
```

Get a description of this composite stage.

```python
class nemo_curator.stages.text.io.reader.parquet.ParquetReaderStage(
    fields: list[str] | None = None,
    read_kwargs: dict[str, typing.Any] = dict(),
    name: str = 'parquet_reader',
    _generate_ids: bool = False,
    _assign_ids: bool = False,
    allow_empty: bool = False
)
```

Dataclass

**Bases:** [BaseFileReader](/nemo-curator/nemo_curator/stages/text/io/reader/base#nemo_curator-stages-text-io-reader-base-BaseFileReader)

Stage that processes a group of Parquet files into a DocumentBatch.
This stage accepts FileGroupTasks created by FilePartitioningStage
and reads the actual file contents into DocumentBatches.

**Parameters:**

**`fields`** `list[str]` — default: None

If specified, only read these columns. Defaults to None.

---

**`read_kwargs`** `dict[str, Any]` — default: dict()

Keyword arguments for the underlying reader. Defaults to \{}.

---

**`name`** `str = 'parquet_reader'`

---

```python
nemo_curator.stages.text.io.reader.parquet.ParquetReaderStage.read_data(
    paths: list[str],
    read_kwargs: dict[str, typing.Any] | None = None,
    fields: list[str] | None = None
) -> pandas.DataFrame
```

Read Parquet files using Pandas. Raises an exception if reading fails.