> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/curator/_mcp/server.

# nemo_curator.utils.lance

## Module Contents

### Functions

| Name                                                                                         | Description                                                    |
| -------------------------------------------------------------------------------------------- | -------------------------------------------------------------- |
| [`add_lance_metadata_columns`](#nemo_curator-utils-lance-add_lance_metadata_columns)         | -                                                              |
| [`encode_lance_blob_columns`](#nemo_curator-utils-lance-encode_lance_blob_columns)           | Rebuild Lance Blob v2 arrays from materialized binary columns. |
| [`materialize_lance_blob_columns`](#nemo_curator-utils-lance-materialize_lance_blob_columns) | Replace scanned Blob v2 descriptors with binary payloads.      |

### Data

[`LANCE_FRAGID_COLUMN`](#nemo_curator-utils-lance-LANCE_FRAGID_COLUMN)

[`LANCE_ROWADDR_COLUMN`](#nemo_curator-utils-lance-LANCE_ROWADDR_COLUMN)

[`LANCE_ROWID_COLUMN`](#nemo_curator-utils-lance-LANCE_ROWID_COLUMN)

### API

```python
nemo_curator.utils.lance.add_lance_metadata_columns(
    table: pyarrow.Table
) -> pyarrow.Table
```

```python
nemo_curator.utils.lance.encode_lance_blob_columns(
    table: pyarrow.Table,
    schema: pyarrow.Schema
) -> pyarrow.Table
```

Rebuild Lance Blob v2 arrays from materialized binary columns.

```python
nemo_curator.utils.lance.materialize_lance_blob_columns(
    dataset: lance.LanceDataset,
    table: pyarrow.Table
) -> pyarrow.Table
```

Replace scanned Blob v2 descriptors with binary payloads.

```python
nemo_curator.utils.lance.LANCE_FRAGID_COLUMN = '__lance_fragid'
```

```python
nemo_curator.utils.lance.LANCE_ROWADDR_COLUMN = '__lance_rowaddr'
```

```python
nemo_curator.utils.lance.LANCE_ROWID_COLUMN = '__lance_rowid'
```