`stages.video.io.video_reader`#

Module Contents#

Classes#

`VideoReader`	Composite stage that reads video files from storage and downloads/processes them.
`VideoReaderStage`	Stage that reads video files from local filesystem and extracts metadata.

API#

class stages.video.io.video_reader.VideoReader#

Bases: nemo_curator.stages.base.CompositeStage[nemo_curator.tasks._EmptyTask, nemo_curator.tasks.video.VideoTask]

Composite stage that reads video files from storage and downloads/processes them.

This stage combines FilePartitioningStage and VideoReaderStage into a single high-level operation for reading video files from a directory and processing them with metadata extraction.

Args: input_video_path: Path to the directory containing video files video_limit: Maximum number of videos to process (None for unlimited) verbose: Whether to enable verbose logging during download/processing

Initialization

decompose() → list[nemo_curator.stages.base.ProcessingStage]#

Decompose into constituent execution stages.

Returns: List of processing stages: [FilePartitioningStage, VideoReaderStage]

get_description() → str#: Get a description of what this composite stage does.

input_video_path: str#: None

verbose: bool#: False

video_limit: int | None#: None

class stages.video.io.video_reader.VideoReaderStage#

Bases: nemo_curator.stages.base.ProcessingStage[nemo_curator.tasks.file_group.FileGroupTask, nemo_curator.tasks.video.VideoTask]

Stage that reads video files from local filesystem and extracts metadata.

This stage processes video files by reading their binary content from the local filesystem and extracting comprehensive metadata including dimensions, frame rate, duration, codecs, and other technical properties. The stage handles both the file I/O operations and metadata extraction, storing results in the VideoTask.

The stage performs the following operations:

Reads video file bytes from the local filesystem
Extracts technical metadata using video analysis tools
Validates metadata completeness and logs warnings for missing fields
Optionally logs detailed video information when verbose mode is enabled

Args: verbose: If True, logs detailed video information after successful processing

input_path: str | None#: None

inputs() → tuple[list[str], list[str]]#

Define the input attributes required by this stage.

Returns: Tuple of (top_level_attrs, data_attrs) where: - top_level_attrs: [“data”] - requires VideoTask.data to be populated

outputs() → tuple[list[str], list[str]]#

Define the output attributes produced by this stage.

Returns: Tuple of (top_level_attrs, data_attrs) where: - top_level_attrs: [“data”] - populates VideoTask.data - data_attrs: [“source_bytes”, “metadata”] - populates Video.source_bytes and Video.metadata

process( task: nemo_curator.tasks.file_group.FileGroupTask, ) → nemo_curator.tasks.video.VideoTask#

Process a video task by reading file bytes and extracting metadata.

Performs the complete video processing workflow including reading the video file from disk, extracting technical metadata, and optionally logging detailed information. Returns the same task with populated data.

Args: task: VideoTask containing a Video object with input_video path set.

Returns: The same VideoTask with video.source_bytes and video.metadata populated. If errors occur, the task is returned with error information stored.

verbose: bool#: False

stages.video.io.video_reader#

Module Contents#

Classes#

API#

`stages.video.io.video_reader`#