Add Custom Environment

Learn how to package dependencies for NeMo Curator using a container image.

The NeMo Curator container provides a primary curator conda environment with pre-installed dependencies. If your pipeline needs additional system or Python packages, create a custom image. Refer to the container environments reference for defaults and build arguments.

Before You Start

Before you begin, make sure that you have:

• Reviewed the pipeline concepts and diagrams.
• A base container image suitable for NeMo Curator.
• Optionally created custom code that defines your new requirements.

How to Add Dependencies with a Dockerfile

Define Build Steps

1. Create an environments directory anywhere on your system to organize your custom pipeline stage environments.

2. Create a new folder for your environment, for example: my-env/.

3. Create a Dockerfile that installs your environment’s dependencies on top of the base image.

   1
   FROM <your-base-image>
   2
   3
   # System deps
   4
   RUN apt-get update && apt-get install -y --no-install-recommends \
   5
       wget git && rm -rf /var/lib/apt/lists/*
   6
   7
   # Python deps (prefer aligning with pyproject optional extras)
   8
   # Example: install video stacks used by NeMo Curator
   9
   RUN pip install --no-cache-dir \
   10
       av==13.1.0 opencv-python einops easydict
   11
   12
   # Optional: CV-CUDA (choose wheel that matches your CUDA/Python)
   13
   # RUN pip install cvcuda_cu12
   14
   15
   # Optional: vLLM / FlashAttention (see platform markers in pyproject)
   16
   # RUN pip install vllm==0.9.2 flash-attn&lt;=2.8.3
   17
   18
   # Copy your code if needed
   19
   COPY . /workspace
   20
   WORKDIR /workspace

4. Save the file.

Build the Container

Build and tag your image using Docker or your preferred tool:

$
docker build -t my-ray-curator:latest .

Next Steps

Now that you have created a custom environment, you can create custom code for that environment.