> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/v25.09/about/concepts/text/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/curator/_mcp/server.

# Text Concepts

## Docs

- [Data Loading Concepts](https://docs.nvidia.com/nemo/curator/v25.09/about/concepts/text/data/loading.md): Core concepts for loading and managing text datasets using pipeline-based readers and DocumentBatch tasks
- [Data Acquisition Concepts](https://docs.nvidia.com/nemo/curator/v25.09/about/concepts/text/data/acquisition.md): Core concepts for acquiring text data from remote sources including DocumentDownloader, DocumentIterator, and DocumentExtractor components
- [Text Processing Concepts](https://docs.nvidia.com/nemo/curator/v25.09/about/concepts/text/data/processing.md): Text processing workflows including quality filtering, fuzzy deduplication, content cleaning, and pipeline design
- [Text Data Curation Pipeline](https://docs.nvidia.com/nemo/curator/v25.09/about/concepts/text/data/data-curation-pipeline.md): Comprehensive overview of NeMo Curator's text curation pipeline architecture including data acquisition and processing


> **Note:** This page contains both a page directory (above) and the landing page content (below). The page directory is generated for agent use and does not appear on the landing page.

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/v25.09/about/concepts/text/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/curator/_mcp/server.

# Text Curation Concepts

> Essential concepts for text data curation including loading and processing.

This document covers the essential concepts for text data curation in NVIDIA NeMo Curator. These concepts assume basic familiarity with data science and machine learning principles.

## Core Concept Areas

Text curation in NeMo Curator focuses on these key areas:

#### [Text Curation Pipeline](/about/concepts/text/data/data-curation-pipeline)

Comprehensive overview of the end-to-end text curation architecture and workflow

#### [Data Loading](/about/concepts/text/data/loading)

Core concepts for loading and managing text datasets from local files

#### [Data Acquisition](/about/concepts/text/data/acquisition)

Components for downloading and extracting data from remote sources

#### [Data Processing](/about/concepts/text/data/processing)

Concepts for filtering, deduplication, and classification

## Infrastructure Components

The text curation concepts build on NVIDIA NeMo Curator's core infrastructure components, which are shared across all modalities. These components include:

#### [Memory Management](/reference/infra/memory-management)

Optimize memory usage when processing large datasets

#### [GPU Acceleration](/reference/infra/gpu-processing)

Leverage NVIDIA GPUs for faster data processing

#### [Resumable Processing](/reference/infra/resumable-processing)

Continue interrupted operations across large datasets