Container Environments

NeMo Curator source releases continue, and the repository Dockerfile remains maintained. Starting with the 26.09 release, NVIDIA no longer publishes NeMo Curator container images to NGC. Previously published images remain downloadable. Build an image from a source release or the main development branch.

Build an Image

Build a local Docker image from a source checkout, run it, then activate and verify the Curator environment.

1. Clone the NeMo Curator repository or enter an existing checkout. Use a source release tag available in the repository for released code, or keep main for current development.

2. Build the image from the repository root:

   docker build -f docker/Dockerfile -t nemo-curator:local .

3. Start a container with a shell:

   docker run --gpus all -it --rm \
     -v "$PWD:/workspace" \
     -w /workspace \
     nemo-curator:local bash

4. Activate the virtual environment and verify the import:

   source /opt/venv/env.sh
   python -c "import nemo_curator; print(nemo_curator.__version__)"

   The command prints the installed version and exits without an import error.

Default builds use the all Curator extra, Python 3.13, CUDA 12.9.1, and Ubuntu 24.04. The Dockerfile installs etcd and nats-server for Dynamo and builds the Nemotron OCR C++/CUDA extension. The base image does not include FFmpeg. For audio or video, use the derived image instructions below to package system dependencies with Curator rather than updating each host. Build arguments and Dockerfile revisions can change image contents.

Add FFmpeg for Audio or Video

Build the base image first as described in Build an Image, then create one of these Dockerfiles in the repository root. The audio image uses Ubuntu’s FFmpeg package for audio resampling and PCM/WAV operations. The video image runs Curator’s maintained FFmpeg builder and adds software H.264/HEVC/AV1 decoders while retaining NVENC and libvpx-vp9 support; it omits audio codecs and WAV filters, so use it for video-only workflows.

For audio, save this as Dockerfile.audio:

FROM nemo-curator:local
RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg && rm -rf /var/lib/apt/lists/*

Build the image, then start a shell inside it. Add --gpus all when audio processing will use a GPU; omit that flag for CPU-only audio:

docker build -f Dockerfile.audio -t nemo-curator:audio .
docker run -it --rm \
  -v "$PWD:/workspace" -w /workspace \
  nemo-curator:audio bash

Put input data and desired results under the mounted workspace (or mount their host directories too) so files persist after the container exits.

Run the following smoke commands inside the container. They verify PCM encoding, 16 kHz resampling, and WAV output:

source /opt/venv/env.sh
ffmpeg -v error \
  -f lavfi -i "sine=frequency=1000:duration=0.1" \
  -ar 16000 -ac 1 -acodec pcm_s16le -y \
  /tmp/nemo-curator-audio-smoke.wav
ffprobe -v error \
  -show_entries stream=codec_name,sample_rate,channels \
  -of default=noprint_wrappers=1 \
  /tmp/nemo-curator-audio-smoke.wav
rm /tmp/nemo-curator-audio-smoke.wav

For video, save this as Dockerfile.video:

FROM nemo-curator:local
RUN bash /opt/Curator/docker/common/install_h264_support.sh

Build the image and start a shell inside it:

docker build -f Dockerfile.video -t nemo-curator:video .
docker run --gpus all -it --rm \
  -v "$PWD:/workspace" -w /workspace \
  nemo-curator:video bash

Run these checks inside the container. Confirm that the output lists the NVENC and VP9 encoders and the software H.264 decoder used by CPU-side probing:

source /opt/venv/env.sh
ffmpeg -hide_banner -version | head -n 5
ffmpeg -encoders | grep -E "h264_nvenc|libvpx-vp9"
ffmpeg -decoders | grep -E '^[[:space:]]+V[^[:space:]]+[[:space:]]+h264[[:space:]]'

Keep input videos and output clips under /workspace (or mount their host directories too) so they persist after the container exits. Activate the Curator environment with source /opt/venv/env.sh before running Python commands.

h264_nvenc requires NVIDIA hardware with NVENC support. NVIDIA A100 and H100 GPUs do not include NVENC hardware; use libvpx-vp9 on those GPUs. For Python package installs, the Installation Guide also documents user-local and host-level FFmpeg options.

Use Images on a Cluster

A local Docker tag such as nemo-curator:local is available only to the Docker daemon where it was built. To use the image on a cluster:

1. Push it to a registry reachable from every allocated node, or create a SquashFS image with Enroot and place it on shared storage using these Pyxis instructions. Enroot also supports registry references such as docker://REGISTRY#PATH:TAG; see the Enroot import documentation. Use an image built for the cluster’s CPU architecture. For media jobs, export the corresponding audio or video image tag instead.
2. Set the cluster job’s image reference or path. The SLURM examples use CONTAINER_IMAGE; refer to Multi-Node Ray on SLURM, SLURM Job Arrays, or the SLURM tutorial.

Create a SquashFS Image for Pyxis

On a Linux machine with Enroot and access to the local Docker daemon, export the image and copy the resulting file to storage visible to the cluster:

enroot import --output nemo-curator.sqsh dockerd://nemo-curator:local
mkdir -p /shared/containers
cp nemo-curator.sqsh /shared/containers/nemo-curator.sqsh

On the cluster, confirm that the file is readable and set the path used by the SLURM scripts:

ls -lh /shared/containers/nemo-curator.sqsh
export CONTAINER_IMAGE=/shared/containers/nemo-curator.sqsh

For a media image, replace nemo-curator:local with the appropriate local image tag before importing it. Ensure the image architecture matches the cluster nodes.

Image Configuration

The current Dockerfile accepts these build-time arguments:

These defaults apply when you build the matching Dockerfile revision without overriding its arguments.

Architecture Notes

Python dependencies and extras can depend on the target architecture in pyproject.toml. Some audio dependencies support only x86_64 Linux. Check the package markers before building for arm64.

Ray executes NeMo Curator pipelines.