nemo_curator.stages.audio.metrics.wer

WER / CER computation stage.

Module Contents

Classes

Name	Description
`ComputeWERStage`	Stage that computes Word Error Rate (WER), CER, edge CER, and optionally PNC WER/CER.
`GetPairwiseWerStage`	Compute pairwise word-error-rate (WER) as a percentage for each pair of text and pred_text.

API

class nemo_curator.stages.audio.metrics.wer.ComputeWERStage(
    language: str = 'en',
    hypothesis_text_key: str = 'text',
    reference_text_key: str = 'text_ref',
    num_words_threshold: int = 200,
    num_words_look_back: int = 5,
    compute_pnc_wer: bool = False,
    pnc_chars: str = '،؟.、？¿!,?।',
    edge_length: int = 12,
    segments_key: str = 'segments',
    name: str = 'ComputeWER',
    _normalizer: typing.Any = None
)

Dataclass

Bases: ProcessingStage[AudioTask, AudioTask]

Stage that computes Word Error Rate (WER), CER, edge CER, and optionally PNC WER/CER. This stage cleans the text and normalizes it using NeMo text processing (numbers to words, etc).

Operates on segments within each entry (audio_segment[“hypothesis_text_key”] vs audio_segment[“reference_text_key”]). If “segments” is not in the data entry, the stage will compute WER, CER, edge CER, and optionally PNC WER/CER for the entire entry.

Parameters:

language

strDefaults to 'en'

Language of the text. Defaults to “en”.

hypothesis_text_key

strDefaults to 'text'

Key to the hypothesis text. Defaults to “text”.

reference_text_key

strDefaults to 'text_ref'

Key to the reference text. Defaults to “text”.

num_words_threshold

intDefaults to 200

Number of words to use for normalization. Defaults to 200.

num_words_look_back

intDefaults to 5

Number of words to look back for normalization. Defaults to 5.

compute_pnc_wer

boolDefaults to False

Whether to compute PNC WER/CER. Defaults to False.

pnc_chars

strDefaults to '،؟.、？¿!,?।'

Punctuation characters to use for normalization. Defaults to special punctuation string.

edge_length

intDefaults to 12

Length of the edge to compute CER. Defaults to 12.

segments_key

strDefaults to 'segments'

Key for the segments in the manifest. Defaults to “segments”.

Returns:

The same data as in the input data, but with WER, CER, edge CER, and optionally PNC WER/CER added to each segment.

_normalizer

Any = field(default=None, repr=False)

compute_pnc_wer

bool = False

edge_length

int = 12

hypothesis_text_key

str = 'text'

language

str = 'en'

name

str = 'ComputeWER'

num_words_look_back

int = 5

num_words_threshold

int = 200

pnc_chars

str = '،؟.、？¿!,?।'

reference_text_key

str = 'text_ref'

segments_key

str = 'segments'

nemo_curator.stages.audio.metrics.wer.ComputeWERStage.__post_init__() -> None

nemo_curator.stages.audio.metrics.wer.ComputeWERStage.clean_text(
    text: str,
    retain_pncs: bool = True
) -> str

Clean text by removing invalid characters.

nemo_curator.stages.audio.metrics.wer.ComputeWERStage.get_char_rate(
    text: str,
    duration: float
) -> float

Calculate character rate (chars per second).

nemo_curator.stages.audio.metrics.wer.ComputeWERStage.get_wer(
    audio_segment: dict[str, typing.Any]
) -> None

Compute WER, CER, edge CER, and optionally PNC WER/CER per segment.

nemo_curator.stages.audio.metrics.wer.ComputeWERStage.get_word_rate(
    text: str,
    duration: float
) -> float

Calculate word rate (words per second).

nemo_curator.stages.audio.metrics.wer.ComputeWERStage.inputs() -> tuple[list[str], list[str]]

nemo_curator.stages.audio.metrics.wer.ComputeWERStage.normalize_and_clean_text(
    text: str
) -> tuple[str, str]

Normalize and clean text. Returns (cleaned_with_punct, cleaned_without_punct).

nemo_curator.stages.audio.metrics.wer.ComputeWERStage.normalize_text(
    text: str
) -> str

Normalize text using NeMo text processing (numbers to words, etc).

nemo_curator.stages.audio.metrics.wer.ComputeWERStage.outputs() -> tuple[list[str], list[str]]

nemo_curator.stages.audio.metrics.wer.ComputeWERStage.process(
    task: nemo_curator.tasks.AudioTask
) -> nemo_curator.tasks.AudioTask

Compute WER, CER, edge CER, and optionally PNC WER/CER per segment.

nemo_curator.stages.audio.metrics.wer.ComputeWERStage.setup(
    _worker_metadata: nemo_curator.backends.base.WorkerMetadata | None = None
) -> None

Setup stage.

nemo_curator.stages.audio.metrics.wer.ComputeWERStage.strip_spaces_before_punctuations(
    text: str
) -> str

Strip spaces before punctuation characters.

nemo_curator.stages.audio.metrics.wer.ComputeWERStage.validate_input(
    task: nemo_curator.tasks.AudioTask
) -> bool

OR-shaped validation: segments OR top-level text keys must be present.

class nemo_curator.stages.audio.metrics.wer.GetPairwiseWerStage(
    name: str = 'GetPairwiseWerStage',
    text_key: str = 'text',
    pred_text_key: str = 'pred_text',
    wer_key: str = 'wer_pct'
)

Dataclass

Bases: ProcessingStage[AudioTask, AudioTask]

Compute pairwise word-error-rate (WER) as a percentage for each pair of text and pred_text.

WER is measured between data[self.text_key] and data[self.pred_text_key] and stored as a percentage (e.g. 5.0 means 5% WER).

Parameters:

text_key

strDefaults to 'text'

Key for the utterance transcript. Defaults to “text”.

pred_text_key

strDefaults to 'pred_text'

Key for the ASR predictions. Defaults to “pred_text”.

wer_key

strDefaults to 'wer_pct'

Key to store the computed WER percentage. Defaults to “wer_pct”.

name

str = 'GetPairwiseWerStage'

pred_text_key

str = 'pred_text'

text_key

str = 'text'

wer_key

str = 'wer_pct'

nemo_curator.stages.audio.metrics.wer.GetPairwiseWerStage.inputs() -> tuple[list[str], list[str]]

nemo_curator.stages.audio.metrics.wer.GetPairwiseWerStage.outputs() -> tuple[list[str], list[str]]

nemo_curator.stages.audio.metrics.wer.GetPairwiseWerStage.process(
    task: nemo_curator.tasks.AudioTask
) -> nemo_curator.tasks.AudioTask

Compute WER percentage between hypothesis and reference text.