> This page is for version 26.07 · v1.3.0.
> For other versions, use one of these documentation indexes:
> - Latest · v1.4.0 (26.09) (default): https://docs.nvidia.com/nemo/curator/latest/llms.txt
> - Main · preview: https://docs.nvidia.com/nemo/curator/main/llms.txt
> - 26.09 · v1.4.0: https://docs.nvidia.com/nemo/curator/v26.09/llms.txt
> - 26.07 · v1.3.0: https://docs.nvidia.com/nemo/curator/v26.07/llms.txt
> - 26.04 · v1.2.0: https://docs.nvidia.com/nemo/curator/v26.04/llms.txt
> - 26.02 · v1.1.0: https://docs.nvidia.com/nemo/curator/v26.02/llms.txt
> - 25.09 · v1.0.0: https://docs.nvidia.com/nemo/curator/v25.09/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/curator/_mcp/server.

# nemo_curator.utils.column_utils

## Module Contents

### Functions

| Name                                                                                                | Description                                                            |
| --------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- |
| [`pyarrow_string_to_pandas_dtype`](#nemo_curator-utils-column_utils-pyarrow_string_to_pandas_dtype) | Map Arrow strings to a pandas dtype with functional string operations. |
| [`pyarrow_to_pandas_dtype`](#nemo_curator-utils-column_utils-pyarrow_to_pandas_dtype)               | Map Arrow types to pandas dtypes with functional string operations.    |
| [`resolve_filename_column`](#nemo_curator-utils-column_utils-resolve_filename_column)               | Resolve the filename column name based on the input parameter.         |

### API

```python
nemo_curator.utils.column_utils.pyarrow_string_to_pandas_dtype(
    arrow_type: pyarrow.DataType,
    na_value: object = pd.NA
) -> pandas.StringDtype | None
```

Map Arrow strings to a pandas dtype with functional string operations.

```python
nemo_curator.utils.column_utils.pyarrow_to_pandas_dtype(
    arrow_type: pyarrow.DataType
) -> pandas.StringDtype | pandas.ArrowDtype
```

Map Arrow types to pandas dtypes with functional string operations.

```python
nemo_curator.utils.column_utils.resolve_filename_column(
    add_filename_column: bool | str
) -> str | None
```

Resolve the filename column name based on the input parameter.

**Parameters:**

**`add_filename_column`** `bool | str`

Can be:

* True: Use default filename column name
* False: No filename column
* str: Use the provided string as column name

---

**Returns:** `str | None`

str | None: The filename column name or None if not needed