> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/v25.09/curate-audio/process-data/quality-assessment/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/curator/_mcp/server.

# Quality Assessment

## Docs

- [WER Filtering](https://docs.nvidia.com/nemo/curator/v25.09/curate-audio/process-data/quality-assessment/wer-filtering.md): Filter audio samples based on Word Error Rate thresholds to ensure high-quality transcription accuracy
- [Duration Filtering](https://docs.nvidia.com/nemo/curator/v25.09/curate-audio/process-data/quality-assessment/duration-filtering.md): Filter audio samples by duration ranges, speech rate metrics, and temporal characteristics for optimal dataset quality


> **Note:** This page contains both a page directory (above) and the landing page content (below). The page directory is generated for agent use and does not appear on the landing page.

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/v25.09/curate-audio/process-data/quality-assessment/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/curator/_mcp/server.

# Quality Assessment for Audio Data

> Assess and filter audio quality using transcription accuracy metrics, duration analysis, and custom quality measures

Filter audio quality using transcription accuracy metrics, duration analysis, and custom quality measures to ensure high-quality speech datasets for ASR training.

## How it Works

Audio quality assessment in NeMo Curator focuses on speech-specific metrics that correlate with training data quality:

1. **Transcription Accuracy**: Word Error Rate (WER) and Character Error Rate (CER) between ground truth and ASR predictions
2. **Duration Analysis**: Audio length validation and speech rate calculations
3. **Value-based Filtering**: Configurable filtering using comparison operators

## Quality Metrics

### Word Error Rate (WER)

The primary metric for assessing transcription quality:

```python
from nemo_curator.stages.audio.metrics.get_wer import GetPairwiseWerStage

# Calculate WER for each audio sample
wer_stage = GetPairwiseWerStage(
    text_key="text",           # Ground truth transcription
    pred_text_key="pred_text", # ASR prediction
    wer_key="wer"             # Output WER field
)
```

WER measures the percentage of words that differ between ground truth and predicted transcriptions:

* **WER = 0%**: Perfect transcription match
* **WER = 25%**: Good quality (1 in 4 words incorrect)
* **WER = 50%**: Moderate quality
* **WER > 75%**: Poor quality (consider filtering)

### Character Error Rate (CER)

More granular accuracy measurement at the character level. The `get_cer()` function is a utility for calculating CER programmatically:

```python
from nemo_curator.stages.audio.metrics.get_wer import get_cer

# Calculate CER between two strings (for testing/validation)
cer_value = get_cer("hello world", "helo world")  # Returns 9.09
```

> **Note**
>
> The WER and CER utilities depend on the `editdistance` package. These are utility functions typically used within custom stages rather than directly in pipelines.

### Speech Rate Metrics

NeMo Curator provides utility functions for analyzing speaking speed and content density. These functions are designed for use in custom processing stages:

```python
from nemo_curator.stages.audio.metrics.get_wer import get_wordrate, get_charrate

# Calculate words per second
word_rate = get_wordrate("hello world example", 2.5)  # 1.2 words/second

# Calculate characters per second
char_rate = get_charrate("hello world", 2.0)  # 5.5 chars/second
```

For a complete example of using speech rate metrics in a pipeline, refer to the [Duration Filtering](/curate-audio/process-data/quality-assessment/duration-filtering) guide.

## Filtering Strategies

### WER-based Filtering

Filter audio samples based on transcription accuracy:

```python
from nemo_curator.stages.audio.common import PreserveByValueStage

# Keep samples with WER <= 30% (high quality)
high_quality_filter = PreserveByValueStage(
    input_value_key="wer",
    target_value=30.0,
    operator="le"  # less than or equal
)

# Remove samples with WER >= 80% (very poor quality)
poor_quality_filter = PreserveByValueStage(
    input_value_key="wer",
    target_value=80.0,
    operator="lt"  # less than
)
# Preserves only entries with WER < 80%
```

### Duration-based Filtering

Filter by audio length to remove short or long samples:

```python
from nemo_curator.stages.audio.common import PreserveByValueStage

# Keep samples between 1-30 seconds
duration_min_filter = PreserveByValueStage(
    input_value_key="duration",
    target_value=1.0,
    operator="ge"  # greater than or equal
)

duration_max_filter = PreserveByValueStage(
    input_value_key="duration",
    target_value=30.0,
    operator="le"  # less than or equal
)
```

### Combined Quality Filtering

```python
from nemo_curator.pipeline import Pipeline
from nemo_curator.stages.audio.metrics.get_wer import GetPairwiseWerStage
from nemo_curator.stages.audio.common import GetAudioDurationStage, PreserveByValueStage

# Create multi-stage quality pipeline
quality_pipeline = Pipeline(name="audio_quality_assessment")

# Calculate all metrics
quality_pipeline.add_stage(GetPairwiseWerStage())
quality_pipeline.add_stage(GetAudioDurationStage(
    audio_filepath_key="audio_filepath",
    duration_key="duration"
))

# Apply filters in sequence
filters = [
    PreserveByValueStage("wer", 50.0, "le"),        # WER <= 50%
    PreserveByValueStage("duration", 1.0, "ge"),     # Duration >= 1s
    PreserveByValueStage("duration", 20.0, "le"),    # Duration <= 20s
]

for filter_stage in filters:
    quality_pipeline.add_stage(filter_stage)
```

## Operator Options

The `PreserveByValueStage` supports several comparison operators:

| Operator | Description           | Example Use Case        |
| -------- | --------------------- | ----------------------- |
| `"eq"`   | Equal to              | Exact duration matching |
| `"ne"`   | Not equal to          | Exclude specific values |
| `"lt"`   | Less than             | Max thresholds          |
| `"le"`   | Less than or equal    | Quality thresholds      |
| `"gt"`   | Greater than          | Min thresholds          |
| `"ge"`   | Greater than or equal | Min requirements        |

## Complete Quality Assessment Pipeline

Here's a complete working example that demonstrates quality assessment:

```python
from nemo_curator.pipeline import Pipeline
from nemo_curator.backends.xenna import XennaExecutor
from nemo_curator.stages.audio.datasets.fleurs.create_initial_manifest import CreateInitialManifestFleursStage
from nemo_curator.stages.audio.inference.asr_nemo import InferenceAsrNemoStage
from nemo_curator.stages.audio.metrics.get_wer import GetPairwiseWerStage
from nemo_curator.stages.audio.common import GetAudioDurationStage, PreserveByValueStage
from nemo_curator.stages.audio.io.convert import AudioToDocumentStage
from nemo_curator.stages.text.io.writer import JsonlWriter
from nemo_curator.stages.resources import Resources

# Create complete quality assessment pipeline
pipeline = Pipeline(name="audio_quality_assessment")

# 1. Load data
pipeline.add_stage(CreateInitialManifestFleursStage(
    lang="hy_am",
    split="dev",
    raw_data_dir="./audio_data"
).with_(batch_size=4))

# 2. ASR inference
pipeline.add_stage(InferenceAsrNemoStage(
    model_name="nvidia/stt_hy_fastconformer_hybrid_large_pc"
).with_(resources=Resources(gpus=1.0)))

# 3. Calculate quality metrics
pipeline.add_stage(GetPairwiseWerStage())
pipeline.add_stage(GetAudioDurationStage(
    audio_filepath_key="audio_filepath",
    duration_key="duration"
))

# 4. Apply quality filters
pipeline.add_stage(PreserveByValueStage(
    input_value_key="wer",
    target_value=75.0,
    operator="le"  # Keep WER <= 75%
))

pipeline.add_stage(PreserveByValueStage(
    input_value_key="duration",
    target_value=1.0,
    operator="ge"  # Keep duration >= 1s
))

pipeline.add_stage(PreserveByValueStage(
    input_value_key="duration",
    target_value=30.0,
    operator="le"  # Keep duration <= 30s
))

# 5. Export high-quality results
pipeline.add_stage(AudioToDocumentStage())
pipeline.add_stage(JsonlWriter(path="./high_quality_audio"))

# Execute pipeline
executor = XennaExecutor()
pipeline.run(executor)
```

## Related Topics

* [WER Filtering](/curate-audio/process-data/quality-assessment/wer-filtering) - Detailed guide to Word Error Rate filtering
* [Duration Filtering](/curate-audio/process-data/quality-assessment/duration-filtering) - Audio length and speech rate filtering
* [Audio Analysis](/curate-audio/process-data/audio-analysis) - Audio file analysis and validation