API ReferenceFull Library ReferenceNemo CuratorNemo CuratorStagesVideoIo

nemo_curator.stages.video.io.video_reader

View as Markdown

Module Contents

Classes

NameDescription
VideoReaderComposite stage that reads video files from storage and downloads/processes them.
VideoReaderStageStage that reads video files from local filesystem and extracts metadata.

API

class nemo_curator.stages.video.io.video_reader.VideoReader(
    input_video_path: str,
    video_limit: int | None = None,
    verbose: bool = False
)
Dataclass

Bases: CompositeStage[_EmptyTask, VideoTask]

Composite stage that reads video files from storage and downloads/processes them.

This stage combines FilePartitioningStage and VideoReaderStage into a single high-level operation for reading video files from a directory and processing them with metadata extraction.

Parameters:

input_video_path
str

Path to the directory containing video files

video_limit
int | NoneDefaults to None

Maximum number of videos to process (None for unlimited)

verbose
boolDefaults to False

Whether to enable verbose logging during download/processing

input_video_path
str
verbose
bool = False
video_limit
int | None = None
nemo_curator.stages.video.io.video_reader.VideoReader.__post_init__()

Initialize the parent CompositeStage after dataclass initialization.

nemo_curator.stages.video.io.video_reader.VideoReader.decompose() -> list[nemo_curator.stages.base.ProcessingStage]

Decompose into constituent execution stages.

Returns: list[ProcessingStage]

List of processing stages: [FilePartitioningStage, VideoReaderStage]

nemo_curator.stages.video.io.video_reader.VideoReader.get_description() -> str

Get a description of what this composite stage does.

class nemo_curator.stages.video.io.video_reader.VideoReaderStage(
    input_path: str | None = None,
    verbose: bool = False,
    name: str = 'video_reader'
)
Dataclass

Bases: ProcessingStage[FileGroupTask, VideoTask]

Stage that reads video files from local filesystem and extracts metadata.

This stage processes video files by reading their binary content from the local filesystem and extracting comprehensive metadata including dimensions, frame rate, duration, codecs, and other technical properties. The stage handles both the file I/O operations and metadata extraction, storing results in the VideoTask.

The stage performs the following operations:

  1. Reads video file bytes from the local filesystem
  2. Extracts technical metadata using video analysis tools
  3. Validates metadata completeness and logs warnings for missing fields
  4. Optionally logs detailed video information when verbose mode is enabled

Parameters:

verbose
boolDefaults to False

If True, logs detailed video information after successful processing

input_path
str | None = None
name
str = 'video_reader'
verbose
bool = False
nemo_curator.stages.video.io.video_reader.VideoReaderStage._download_video_bytes(
    video: nemo_curator.tasks.video.Video
) -> bool

Read video file bytes from the local filesystem.

Reads the complete binary content of the video file and stores it in the video.source_bytes attribute. Handles file I/O errors gracefully and logs appropriate error messages.

Parameters:

video
Video

Video object containing the input_video path to read from.

Returns: bool

True if file reading was successful, False if an error occurred.

nemo_curator.stages.video.io.video_reader.VideoReaderStage._extract_and_validate_metadata(
    video: nemo_curator.tasks.video.Video
) -> bool

Extract comprehensive metadata from video file and validate completeness.

Uses video analysis tools to extract technical metadata including dimensions, frame rate, duration, codecs, bit rate, and other properties. Logs warnings for critical missing metadata fields like codec and pixel format.

Parameters:

video
Video

Video object with source_bytes populated for metadata extraction.

Returns: bool

True if metadata extraction completed successfully, False if extraction

nemo_curator.stages.video.io.video_reader.VideoReaderStage._format_metadata_for_logging(
    video: nemo_curator.tasks.video.Video
) -> dict[str, str]

Format video metadata into human-readable strings for logging output.

Converts raw metadata values into formatted strings with appropriate units and handles None values gracefully by substituting “unknown” placeholders. Used by _log_video_info for consistent log formatting.

Parameters:

video
Video

Video object with populated metadata fields.

Returns: dict[str, str]

Dictionary mapping metadata field names to formatted string values,

nemo_curator.stages.video.io.video_reader.VideoReaderStage._log_video_info(
    video: nemo_curator.tasks.video.Video
) -> None

Log comprehensive video information after successful processing.

Outputs detailed information about the processed video including file size, resolution, frame rate, duration, weight, and bit rate. This method is only called when verbose mode is enabled.

Parameters:

video
Video

Video object with populated metadata fields.

nemo_curator.stages.video.io.video_reader.VideoReaderStage.inputs() -> tuple[list[str], list[str]]

Define the input attributes required by this stage.

Returns: list[str]

Tuple of (top_level_attrs, data_attrs) where:

nemo_curator.stages.video.io.video_reader.VideoReaderStage.outputs() -> tuple[list[str], list[str]]

Define the output attributes produced by this stage.

Returns: list[str]

Tuple of (top_level_attrs, data_attrs) where:

nemo_curator.stages.video.io.video_reader.VideoReaderStage.process(
    task: nemo_curator.tasks.file_group.FileGroupTask
) -> nemo_curator.tasks.video.VideoTask

Process a video task by reading file bytes and extracting metadata.

Performs the complete video processing workflow including reading the video file from disk, extracting technical metadata, and optionally logging detailed information. Returns the same task with populated data.

Parameters:

task
FileGroupTask

VideoTask containing a Video object with input_video path set.

Returns: VideoTask

The same VideoTask with video.source_bytes and video.metadata populated.