Migrate to NeMo Curator 26.07

Use this checklist to update applications and pipeline configuration for NeMo Curator 26.07 (v1.3.0). For the complete feature and fix list, see the 26.07 release notes.

Migration at a Glance

1. Upgrade Python

NeMo Curator 26.07 requires Python 3.11 or later and supports Python versions below 3.14. Recreate the environment instead of upgrading Python in place, then install only the extras your pipeline uses.

$
uv venv --python 3.11
$
source .venv/bin/activate
$
uv pip install "nemo-curator[all]"

If you do not use uv, create the environment with venv instead:

$
python3.11 -m venv .venv
$
source .venv/bin/activate
$
python -m pip install --upgrade pip
$
python -m pip install "nemo-curator[all]"

Replace all with a narrower modality extra when possible.

2. Stop Supplying task_id

Task.task_id is framework-owned in 26.07. It has init=False, so passing it to a task constructor raises an error. NeMo Curator assigns IDs at every stage boundary and preserves deterministic lineage for supported one-to-one, one-to-many, and positionally aligned many-to-many mappings. Ambiguous M-to-K mappings receive a random ID prefixed with r.

Before:

1
batch = DocumentBatch(
2
    task_id="batch-0001",
3
    dataset_name="training",
4
    data=documents,
5
)

After:

1
batch = DocumentBatch(
2
    dataset_name="training",
3
    data=documents,
4
)
5
6
results = pipeline.run(initial_tasks=[batch])
7
print(results[0].task_id)

Do not assign task_id inside a custom stage. If you need a durable business identifier, store it in the task payload or metadata and treat task_id as execution lineage.

This change was introduced in #2036, with stale tutorial constructors removed in #2058.

3. Instantiate Sentinel Tasks

EmptyTask is now a class derived from the payloadless SentinelTask base. Construct it like any other task and use type checks instead of object identity.

Before:

1
results = pipeline.run(initial_tasks=[EmptyTask])
2
3
if task is EmptyTask:
4
    return task

After:

1
from nemo_curator.tasks.sentinels import EmptyTask
2
3
results = pipeline.run(initial_tasks=[EmptyTask()])
4
5
if isinstance(task, EmptyTask):
6
    return task

When initial_tasks is omitted, Pipeline.run() normally creates the empty starting task for you. The class conversion and follow-up fixes landed in #2062 and #2070.

4. Remove the Video Model Preprocessing Option

Video caption models now use their vLLM or Hugging Face preprocessing path. The following interfaces no longer accept model_does_preprocess:

• CaptionPreparationStage
• CaptionGenerationStage
• QwenVL
• split_video_into_windows
• The --captioning-model-does-preprocess CLI option

Before:

1
model = QwenVL(
2
    model_dir="/models",
3
    model_variant="qwen2.5",
4
    caption_batch_size=8,
5
    model_does_preprocess=True,
6
)

After:

1
model = QwenVL(
2
    model_dir="/models",
3
    model_variant="qwen2.5",
4
    caption_batch_size=8,
5
)

Remove the CLI flag from shell scripts and job templates. Keep preprocess_dtype when you need to control preprocessing precision. QwenVL rejects the retired keyword with a targeted error instead of silently forwarding it to vLLM. See #2109.

5. Replace the Unversioned Video Caption Variant

The unversioned qwen caption model variant is no longer accepted. Select the version explicitly in both preparation and generation so the prepared-input and caption keys continue to match.

Before:

1
CaptionPreparationStage(model_variant="qwen")
2
CaptionGenerationStage(model_variant="qwen")

After, preserving the previous Qwen2.5 behavior:

1
CaptionPreparationStage(model_variant="qwen2.5")
2
CaptionGenerationStage(model_variant="qwen2.5")

For the example CLI, replace --captioning-algorithm qwen with --captioning-algorithm qwen2.5. To adopt Qwen3-VL instead, use qwen3 consistently. If CaptionEnhancementStage reads the earlier caption, update its captioning_model_variant to the same value. See Captions and Preview for all supported Qwen and Nemotron variants.

6. Replace InferenceModelConfig

InferenceModelConfig was replaced by typed model and backend configuration. Existing Ray Serve deployments should use RayServeModelConfig:

1
from nemo_curator.core.serve import InferenceServer, RayServeModelConfig
2
3
model = RayServeModelConfig(
4
    model_identifier="google/gemma-3-27b-it",
5
    deployment_config={
6
        "autoscaling_config": {"min_replicas": 1, "max_replicas": 1},
7
    },
8
    engine_kwargs={"tensor_parallel_size": 4},
9
)
10
server = InferenceServer(models=[model])

To use NVIDIA Dynamo instead, configure the model and backend separately:

1
from nemo_curator.core.serve import (
2
    DynamoServerConfig,
3
    DynamoVLLMModelConfig,
4
    InferenceServer,
5
)
6
7
model = DynamoVLLMModelConfig(
8
    model_identifier="google/gemma-3-27b-it",
9
    num_replicas=1,
10
    engine_kwargs={"tensor_parallel_size": 4},
11
)
12
server = InferenceServer(models=[model], backend=DynamoServerConfig())

Do not pass Ray Serve’s deployment_config to DynamoVLLMModelConfig. Use num_replicas for aggregated serving or configure the Dynamo prefill and decode roles for disaggregated serving. See Inference Server for the backend decision and complete configuration reference.

7. Separate ASR DataLoader Workers from Stage Workers

In audio ASR stages, the old num_workers constructor argument is now dataloader_num_workers. The default remains 10. The num_workers name is reserved for backend execution workers and is set with with_().

Before:

1
aligner = NeMoASRAlignerStage(
2
    model_name="nvidia/parakeet-tdt-0.6b-v2",
3
    num_workers=4,
4
)

After:

1
aligner = NeMoASRAlignerStage(
2
    model_name="nvidia/parakeet-tdt-0.6b-v2",
3
    dataloader_num_workers=4,
4
).with_(num_workers=2)

The same rename applies to stages derived from BaseASRProcessorStage. The two values control different layers:

• dataloader_num_workers controls loading within each ASR stage worker.
• .with_(num_workers=...) controls how many stage workers the backend schedules.

See #2086 and Stage worker sizing.

8. Review Architecture-Specific Dependencies

Dependency Matrix

Audio Text Processing

In #2049, the audio dependency marker for nemo_text_processing was limited to x86_64 non-macOS systems because pynini wheels are unavailable on the other supported platforms.

On aarch64 or macOS, audit imports of stages such as:

1
from nemo_curator.stages.audio.tagging import InverseTextNormalizationStage
2
from nemo_curator.stages.audio.metrics import ComputeWERStage

If they require nemo_text_processing in your pipeline, run that portion on a supported x86_64 environment or replace the stage. Installing the audio extra alone no longer makes those imports available on aarch64.

FlashAttention

In #2107, NeMo Curator removed the direct flash-attn<=2.8.3 pin from video_cuda12, the lock file, and related Docker build handling. The built-in video stack should run with its declared dependencies. If your own model or stage imports flash_attn, install a version compatible with your CUDA, PyTorch, compiler, and architecture instead of relying on the Curator extra.

9. Configure Pipeline Resumability

NeMo Curator 26.07 includes source-level counter checkpointing from #2063. On rerun, it skips source partitions whose downstream work completed and retries sources with unfinished or failed branches.

Start the Ray cluster before running the pipeline, then pass a local directory to checkpoint_path:

1
from nemo_curator.backends.ray_data import RayDataExecutor
2
from nemo_curator.core.client import RayClient
3
4
client = RayClient()
5
client.start()
6
executor = RayDataExecutor()
7
8
try:
9
    results = pipeline.run(
10
        executor=executor,
11
        checkpoint_path="/shared/checkpoints/my_pipeline",
12
    )
13
finally:
14
    client.stop()

Follow these constraints:

• checkpoint_path must be a local or shared filesystem directory, not an object-store or cloud URI. State is written under .nemo_curator_metadata.
• A Ray cluster must already be running and discoverable through RAY_ADDRESS. RayClient().start() or the SLURM Ray client establishes it.
• Every stage must have is_resumable=True. Pipeline.run() fails before execution if any stage is marked unsafe; do not override the flag until you have verified the stage’s input-to-output mapping.
• Multiple runs, including SLURM array shards, can share the directory because each writer owns a separate LMDB file.
• Keep the checkpoint directory between attempts. Delete it only when intentionally starting the pipeline from scratch.

NoneTask and FailedTask are now released sentinel classes used by the resumability layer. A returned None becomes NoneTask and consumes that branch. FailedTask leaves the source pending so it is retried. Most user stages should continue returning their normal task type or None rather than constructing these markers directly.

See Resumable Processing for the full execution model and supported patterns.

10. Run Post-Upgrade Checks

Use a representative subset rather than a synthetic no-op pipeline. Confirm all of the following:

• Task constructors do not receive task_id, and returned task IDs are populated.
• Explicit empty inputs use EmptyTask().
• Video commands contain no model_does_preprocess option.
• Video caption stages use an explicit, matching model variant rather than qwen.
• Inference servers use RayServeModelConfig or an explicit Dynamo model and server configuration.
• ASR constructors use dataloader_num_workers; stage concurrency is configured separately.
• Ray Data and Xenna schedule the intended number of workers.
• aarch64 jobs do not import unavailable audio text-processing stages.
• Custom video code does not assume FlashAttention was installed by video_cuda12.
• Resumable runs use a pre-existing Ray cluster, a persistent local checkpoint directory, and only resumability-safe stages.
• Output manifests, metrics, and retry behavior match the previous production baseline.

For backend-specific validation, see Execution Backends.