Install (All Modalities)

This guide covers installing NeMo Curator with support for all modalities and verifying your installation is working correctly. For a single-modality install or a 30-minute walkthrough, start with one of the modality quickstarts instead.

Before You Start

System Requirements

For comprehensive system requirements and production deployment specifications, refer to Production Deployment Requirements.

Quick Start Requirements:

• OS: Ubuntu 24.04/22.04/20.04 (recommended)
• Python: 3.10, 3.11, or 3.12
• Memory: 16GB+ RAM for basic text processing
• GPU (optional): NVIDIA GPU with 16GB+ VRAM for acceleration
• CUDA 12 (required for audio_cuda12, video_cuda12, image_cuda12, and text_cuda12 extras)

Development vs Production

Installation Methods

Choose one of the following installation methods based on your needs:

Install FFmpeg and Encoders (Required for Video)

Curator’s video pipelines rely on FFmpeg for decoding and encoding. If you plan to encode clips (using --transcode-encoder h264_nvenc or --transcode-encoder libvpx-vp9), install FFmpeg with NVENC and libvpx-vp9 support. The maintained install script bundles both.

$
curl -fsSL https://raw.githubusercontent.com/NVIDIA-NeMo/Curator/main/docker/common/install_ffmpeg.sh -o install_ffmpeg.sh
$
chmod +x install_ffmpeg.sh
$
sudo bash install_ffmpeg.sh

$
bash /opt/Curator/docker/common/install_h264_support.sh

Software H.264/HEVC/AV1 Codec Support (Advanced)

Curator’s default FFmpeg build deliberately ships NVDEC-only decoders for h264, hevc, and av1, and excludes software H.264 encoders (libopenh264, libx264, libx265). This keeps the codec footprint tight and routes every H.264/HEVC/AV1 decode through the GPU.

You may need to add software codec support in two cases:

• H.264 inputs in CPU-only pipeline stages. VideoReader and ClipWriter invoke ffprobe from CPU-only Ray actors that can’t see the GPU; they need a software h264/hevc/av1 decoder to extract metadata. Without it you’ll get a SoftwareCodecMissingError pointing back here.
• H.264 software encoding (for example, on GPUs without an NVENC encoder block such as A100 or H100, when VP9 isn’t acceptable).

Option 1: Run the bundled installer inside the container (Recommended)

The repository ships a runtime opt-in script that recompiles FFmpeg with software h264/hevc/av1 decoders enabled, optionally including the libopenh264 encoder. It runs inside an existing container — no image rebuild required.

$
# Inside the container — adds h264/hevc/av1 software decoders only (LGPLv3):
$
bash /opt/Curator/docker/common/install_h264_support.sh
$
$
# Same plus the libopenh264 software h264 ENCODER, so --transcode-encoder=libopenh264 works:
$
bash /opt/Curator/docker/common/install_h264_support.sh --with-libopenh264

The build takes ~5–10 minutes, replaces /usr/local/bin/{ffmpeg,ffprobe} in place, and pins to the same FFmpeg tag as the image build. Script source: docker/common/install_h264_support.sh.

License notice: the default mode adds only FFmpeg-internal decoders (LGPL). With --with-libopenh264 the binary additionally links Cisco’s OpenH264 (BSD-2-Clause + Cisco-distributed binary license — see https://www.openh264.org/BINARY_LICENSE.txt). You are responsible for any license obligations the resulting binaries impose on your distribution.

Option 2: Use the System FFmpeg

If you’re not using the Curator container, most Linux distributions ship FFmpeg with libx264 (and sometimes libopenh264) preinstalled:

$
sudo apt-get install -y ffmpeg
$
ffmpeg -hide_banner -encoders | grep -E "libx264|libopenh264"

Make sure the ffmpeg on your PATH is the one you want — it must shadow Curator’s bundled build.

Option 3: Edit install_ffmpeg.sh and Rebuild the Image

For users distributing customized images, edit docker/common/install_ffmpeg.sh before building the container:

• For software h264/hevc/av1 decoders: append h264,hevc,av1 to the --enable-decoder=... list.
• For libopenh264 encoder: add libopenh264-dev to the apt list, libopenh264 to --enable-encoder=..., and --enable-libopenh264 to the configure flags.
• For libx264 encoder: add libx264-dev to the apt list and --enable-libx264 --enable-gpl to the configure flags. Note that --enable-gpl makes the resulting FFmpeg binary GPL-licensed.

Then rebuild your image.

Use the Encoder in ClipTranscodingStage

libopenh264 is accepted by ClipTranscodingStage out of the box. At setup time, the stage probes the local FFmpeg build and raises a clear error pointing back to this section if the encoder is not actually compiled in. Once your FFmpeg build includes it, just pass:

$
python video_split_clip_example.py ... --transcode-encoder libopenh264

For other custom encoders not in SUPPORTED_ENCODERS (for example, libx264), edit nemo_curator/stages/video/clipping/clip_extraction_stages.py to extend the tuple, and add the encoder name to the --transcode-encoder argparse choices list in tutorials/video/getting-started/video_split_clip_example.py:

1
SUPPORTED_ENCODERS = ("h264_nvenc", "libvpx-vp9", "libopenh264", "libx264")  # add yours

Caveats

• Default options for these encoders are not tuned. ClipTranscodingStage only sets quality presets for h264_nvenc and libvpx-vp9. Other encoders run with FFmpeg defaults, which may produce different quality/size trade-offs than you expect — see Configure encoders for how to pass an explicit bitrate.
• The NeMo Curator team does not test custom encoder configurations. Issues filed against custom encoder builds may be closed.

Package Extras

NeMo Curator provides several installation extras to install only the components you need:

Installation Verification

After installation, verify that NeMo Curator is working correctly:

1. Basic Import Test

1
# Test basic imports
2
import nemo_curator
3
print(f"NeMo Curator version: {nemo_curator.__version__}")
4
5
# Test core modules
6
from nemo_curator.pipeline import Pipeline
7
from nemo_curator.tasks import DocumentBatch
8
print("✓ Core modules imported successfully")

2. GPU Availability Check

If you installed GPU support, verify GPU access:

1
# Check GPU availability
2
try:
3
    import torch
4
    if torch.cuda.is_available():
5
        print(f"✓ GPU available: {torch.cuda.get_device_name(0)}")
6
        print(f"✓ GPU memory: {torch.cuda.get_device_properties(0).total_memory / 1e9:.1f} GB")
7
    else:
8
        print("⚠ No GPU detected")
9
    
10
    # Check cuDF for GPU deduplication
11
    import cudf
12
    print("✓ cuDF available for GPU-accelerated deduplication")
13
except ImportError as e:
14
    print(f"⚠ Some GPU modules not available: {e}")

3. Run a Quickstart Tutorial

Try a modality-specific quickstart to see NeMo Curator in action:

• Text Curation Quickstart - Set up and run your first text curation pipeline
• Audio Curation Quickstart - Get started with audio dataset curation
• Image Curation Quickstart - Curate image-text datasets for generative models
• Video Curation Quickstart - Split, encode, and curate video clips at scale