nemo_curator.stages.audio.io.convert

Module Contents

Classes

Name	Description
`AudioToDocumentStage`	Convert AudioTask entries into DocumentBatch DataFrames.

API

class nemo_curator.stages.audio.io.convert.AudioToDocumentStage()

Bases: ProcessingStage[AudioTask, DocumentBatch]

Convert AudioTask entries into DocumentBatch DataFrames.

Overrides process_batch to aggregate an entire batch of AudioTask objects into a single multi-row DocumentBatch, avoiding the overhead of many single-row DataFrames. Set batch_size to control how many audio entries land in each DataFrame (default 64).

batch_size

int = 64

name

= 'AudioToDocumentStage'

nemo_curator.stages.audio.io.convert.AudioToDocumentStage.process(
    task: nemo_curator.tasks.AudioTask
) -> nemo_curator.tasks.DocumentBatch

nemo_curator.stages.audio.io.convert.AudioToDocumentStage.process_batch(
    tasks: list[nemo_curator.tasks.AudioTask]
) -> list[nemo_curator.tasks.DocumentBatch]