nemo_automodel.components.datasets.multimodal.video

Frame sampling for video-containing VLM samples.

Module Contents

Classes

Name	Description
`FrameSampler`	Callable that returns a list of PIL frames for a given file path.

Functions

Name	Description
`extract_frame_number`	Extract the numeric frame suffix used by BAGEL frame-folder samples.
`get_frame_indices`	Select frame indices from a video according to BAGEL sampling mode.
`read_frames_decord`	Read sampled frames from a video file with decord.
`read_frames_folder`	Read sampled frames from a directory of extracted frame images.
`sort_frames`	Sort frame paths by their numeric frame suffix.

API

class nemo_automodel.components.datasets.multimodal.video.FrameSampler(
    max_num_frames = -1,
    min_num_frames = 8,
    sample = 'rand'
)

Callable that returns a list of PIL frames for a given file path.

nemo_automodel.components.datasets.multimodal.video.FrameSampler.__call__(
    file_name
)

nemo_automodel.components.datasets.multimodal.video.extract_frame_number(
    filename
)

Extract the numeric frame suffix used by BAGEL frame-folder samples.

nemo_automodel.components.datasets.multimodal.video.get_frame_indices(
    num_frames,
    vlen,
    sample = 'rand',
    fix_start = None,
    input_fps = 1,
    max_num_frames = -1
)

Select frame indices from a video according to BAGEL sampling mode.

nemo_automodel.components.datasets.multimodal.video.read_frames_decord(
    video_path,
    num_frames,
    sample = 'rand',
    fix_start = None,
    clip = None,
    min_num_frames = 4
)

Read sampled frames from a video file with decord.

nemo_automodel.components.datasets.multimodal.video.read_frames_folder(
    video_path,
    num_frames,
    sample = 'rand',
    fix_start = None,
    min_num_frames = 4
)

Read sampled frames from a directory of extracted frame images.

nemo_automodel.components.datasets.multimodal.video.sort_frames(
    frame_paths
)

Sort frame paths by their numeric frame suffix.