> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/v25.09/curate-video/process-data/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/curator/_mcp/server.

# Process Data

## Docs

- [Video Clipping](https://docs.nvidia.com/nemo/curator/v25.09/curate-video/process-data/clipping.md): Split videos into clips using fixed stride or TransNetV2 scene-change detection
- [Clip Encoding](https://docs.nvidia.com/nemo/curator/v25.09/curate-video/process-data/transcoding.md): Transcode clips with CPU or GPU encoders and tune for quality and performance
- [Filtering](https://docs.nvidia.com/nemo/curator/v25.09/curate-video/process-data/filtering.md): Filter clips by motion or aesthetics to improve dataset quality
- [Embeddings](https://docs.nvidia.com/nemo/curator/v25.09/curate-video/process-data/embeddings.md): Generate clip-level embeddings using Cosmos-Embed1 or InternVideo2
- [Duplicate Removal](https://docs.nvidia.com/nemo/curator/v25.09/curate-video/process-data/dedup.md): Remove near-duplicate clips using semantic clustering and pairwise similarity
- [Frame Extraction](https://docs.nvidia.com/nemo/curator/v25.09/curate-video/process-data/frame-extraction.md): Extract frames from clips or full videos for embeddings, filtering, and analysis
- [Captions and Preview](https://docs.nvidia.com/nemo/curator/v25.09/curate-video/process-data/captions-preview.md): Generate clip captions with Qwen and optional preview images


> **Note:** This page contains both a page directory (above) and the landing page content (below). The page directory is generated for agent use and does not appear on the landing page.

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/v25.09/curate-video/process-data/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/curator/_mcp/server.

# Process Data

> Process video data by splitting into clips, encoding, generating embeddings and captions, and removing duplicates

Use NeMo Curator stages to split videos into clips, encode them, generate embeddings or captions, and remove duplicates.

## How it Works

Create a `Pipeline` and add stages for clip extraction, optional re-encoding and filtering, embeddings or captions, previews, and writing outputs. Each stage is modular and configurable to match your quality and performance needs.

## Processing Options

Choose from the following stages to split, encode, filter, embed, caption, preview, and remove duplicates in your videos:

#### [Clip Videos](/curate-video/process-data/clipping)

Split long videos into shorter clips using fixed stride or scene-change detection.

#### [Encode Clips](/curate-video/process-data/transcoding)

Encode clips to H.264 using CPU or GPU encoders and tune performance.

#### [Filter Clips and Frames](/curate-video/process-data/filtering)

Apply motion-based filtering and aesthetic filtering to improve dataset quality.

#### [Extract Frames](/curate-video/process-data/frame-extraction)

Extract frames from clips or full videos for embeddings, filtering, and analysis.

#### [Create Embeddings](/curate-video/process-data/embeddings)

Generate clip-level embeddings with InternVideo2 or Cosmos-Embed1 for search and duplicate removal.

#### [Create Captions & Preview](/curate-video/process-data/captions-preview)

Produce clip captions and optional preview images for review workflows.

#### [Remove Duplicate Embeddings](/curate-video/process-data/dedup)

Remove near-duplicates using semantic clustering and similarity with generated embeddings.

## Write Outputs

Persist clips, embeddings, previews, and metadata at the end of the pipeline using `ClipWriterStage`. Refer to [Save & Export](/curate-video/save-export) for directory layout and examples.

Example (place as the final stage):

```python
from nemo_curator.stages.video.io.clip_writer import ClipWriterStage

pipeline.add_stage(
    ClipWriterStage(
        output_path=OUT_DIR,
        input_path=VIDEO_DIR,
        upload_clips=True,
        dry_run=False,
        generate_embeddings=True,
        generate_previews=False,
        generate_captions=False,
        embedding_algorithm="internvideo2",
        caption_models=[],
        enhanced_caption_models=[],
        verbose=True,
    )
)
```

Path helpers are available to resolve common locations (such as `clips/`, `filtered_clips/`, `previews/`, `metas/v0/`, and `iv2_embd_parquet/`).