`core.models.audio.nemo_audio_preprocessing`#

Pure-PyTorch mel-spectrogram feature extractor for the NeMo audio encoder.

This module exists to avoid taking a dependency on https://github.com/NVIDIA-NeMo/Speech (and its transitive deps: Lightning, Hydra, librosa, NeMo neural types). The audio model only needs the AudioToMelSpectrogramPreprocessor feature extraction, so we reimplement it here using only stdlib + PyTorch ops, mirroring the NeMo ASR class behavior.

Feature parity with NeMo/Speech’s original feature extractor is demonstrated by the companion upstream PR https://github.com/NVIDIA-NeMo/Speech/pull/15692, which validates that this pure-torch implementation produces matching outputs.

Module Contents#

Classes#

AudioToMelSpectrogramPreprocessor

Standalone PyTorch implementation of NeMo’s log-mel ASR preprocessor.

Functions#

`_hz_to_mel`	Slaney mel conversion matching librosa with htk=False.
`_mel_to_hz`	Inverse Slaney mel conversion matching librosa with htk=False.
`_mel_frequencies`
`_normalize_filterbank`
`_create_mel_filterbank`	Create a mel filter bank equivalent to librosa.filters.mel(…, htk=False).
`normalize_batch`	Normalize features per the given normalize_type, respecting per-sample seq_len.
`splice_frames`	Concatenate `frame_splicing` time-shifted copies of `x` along the feature dim.

Data#

CONSTANT

API#

core.models.audio.nemo_audio_preprocessing.CONSTANT#: 1e-05

core.models.audio.nemo_audio_preprocessing._hz_to_mel(frequencies: torch.Tensor) → torch.Tensor#: Slaney mel conversion matching librosa with htk=False.

core.models.audio.nemo_audio_preprocessing._mel_to_hz(mels: torch.Tensor) → torch.Tensor#: Inverse Slaney mel conversion matching librosa with htk=False.

core.models.audio.nemo_audio_preprocessing._mel_frequencies( n_mels: int, fmin: float, fmax: float, ) → torch.Tensor#

core.models.audio.nemo_audio_preprocessing._normalize_filterbank( filterbank: torch.Tensor, norm: Optional[Union[str, float]], ) → torch.Tensor#

core.models.audio.nemo_audio_preprocessing._create_mel_filterbank( sample_rate: int, n_fft: int, n_mels: int, fmin: float, fmax: float, norm: Optional[Union[str, float]], ) → torch.Tensor#: Create a mel filter bank equivalent to librosa.filters.mel(…, htk=False).

core.models.audio.nemo_audio_preprocessing.normalize_batch( x: torch.Tensor, seq_len: torch.Tensor, normalize_type, )#: Normalize features per the given normalize_type, respecting per-sample seq_len.

core.models.audio.nemo_audio_preprocessing.splice_frames(x: torch.Tensor, frame_splicing: int) → torch.Tensor#: Concatenate frame_splicing time-shifted copies of x along the feature dim.

class core.models.audio.nemo_audio_preprocessing.AudioToMelSpectrogramPreprocessor(

sample_rate=16000,

window_size=0.02,

window_stride=0.01,

n_window_size=None,

n_window_stride=None,

window='hann',

normalize='per_feature',

n_fft=None,

preemph=0.97,

features=64,

lowfreq=0,

highfreq=None,

log=True,

log_zero_guard_type='add',

log_zero_guard_value=2**-24,

dither=1e-05,

pad_to=16,

frame_splicing=1,

exact_pad=False,

pad_value=0,

mag_power=2.0,

rng=None,

nb_augmentation_prob=0.0,

nb_max_freq=4000,

mel_norm='slaney',

use_torchaudio: bool = False,

stft_exact_pad=False,

stft_conv=False,

)#

Bases: torch.nn.Module