nemo_curator.stages.video.filtering.motion_vector_backend

Module Contents

Classes

Name	Description
`DecodedData`	Container for decoded video frames containing motion vector data.
`MotionInfo`	Container for motion detection results.
`VideoResolutionTooSmallError`	Exception raised when video resolution is below the minimum required size.

Functions

Name	Description
`check_if_small_motion`	Check if a video has small motion.
`decode_for_motion`	Decode video for motion detection.
`motion_vectors_to_flowfield`	Compute a canonical flow from motion vectors.

Data

_MIN_SIDE_RESOLUTION

API

class nemo_curator.stages.video.filtering.motion_vector_backend.DecodedData(
    frames: list[numpy.typing.NDArray],
    frame_size: torch.Size
)

Dataclass

Container for decoded video frames containing motion vector data.

This class stores a list of decoded frames, each containing motion vector data, and the dimensions of the RGB decoded frame used to construct the flow vector.

frame_size

Size

frames

list[NDArray]

nemo_curator.stages.video.filtering.motion_vector_backend.DecodedData.get_major_size() -> int

Calculate total size in bytes of all frames in the decoded data.

Returns: int

Total size in bytes.

class nemo_curator.stages.video.filtering.motion_vector_backend.MotionInfo(
    is_small_motion: bool,
    per_patch_min_256: float,
    global_mean: float
)

Dataclass

Container for motion detection results.

This class stores the results of motion detection analysis, including:

Whether the video has small motion
The minimum motion value in a 256x256 patch
The global average motion value across the entire videoq

global_mean

float

is_small_motion

bool

per_patch_min_256

float

class nemo_curator.stages.video.filtering.motion_vector_backend.VideoResolutionTooSmallError()

Exception

Bases: Exception

Exception raised when video resolution is below the minimum required size.

This error occurs when either the width or height of the video is less than the minimum resolution threshold required for motion detection.

nemo_curator.stages.video.filtering.motion_vector_backend.check_if_small_motion(
    mv_list: list[numpy.typing.NDArray],
    frame_shape: torch.Size,
    global_mean_threshold: float = 0.00098,
    per_patch_min_256_threshold: float = 1e-06,
    use_gpu: bool = False,
    batch_size: int = 256
) -> nemo_curator.stages.video.filtering.motion_vector_backend.MotionInfo

Check if a video has small motion.

This function checks if a video has small motion by calculating the global mean and per-pixel average motion values.

Parameters:

mv_list

list[npt.NDArray]

List of motion vectors.

frame_shape

torch.Size

Shape of the frame.

global_mean_threshold

floatDefaults to 0.00098

Threshold for global mean motion.

per_patch_min_256_threshold

floatDefaults to 1e-06

Threshold for per-patch minimum motion.

use_gpu

boolDefaults to False

Whether to use GPU for computation.

batch_size

intDefaults to 256

Size of the batch for processing.

Returns: MotionInfo

MotionInfo object containing the results of the motion detection.

nemo_curator.stages.video.filtering.motion_vector_backend.decode_for_motion(
    video: io.BytesIO,
    thread_count: int = 4,
    target_fps: float = 2.0,
    target_duration_ratio: float = 0.5
) -> nemo_curator.stages.video.filtering.motion_vector_backend.DecodedData

Decode video for motion detection.

This function decodes a video stream to extract motion vectors.

Parameters:

video

io.BytesIO

Input video stream as a bytes object.

thread_count

intDefaults to 4

Number of threads to use for decoding.

target_fps

floatDefaults to 2.0

Target frames per second for motion detection.

target_duration_ratio

floatDefaults to 0.5

Ratio of target duration to source duration.

Returns: DecodedData

DecodedData object containing motion vectors and frame dimensions.

nemo_curator.stages.video.filtering.motion_vector_backend.motion_vectors_to_flowfield(
    mvs: torch.Tensor,
    size: list[int],
    flow: torch.Tensor | None = None
) -> torch.Tensor

Compute a canonical flow from motion vectors.

nemo_curator.stages.video.filtering.motion_vector_backend._MIN_SIDE_RESOLUTION = 256