nemo_automodel.components.models.inkling.feature_extraction

Native log-mel feature extraction for Inkling audio inputs.

Module Contents

Classes

Name	Description
`InklingFeatureExtractor`	Extract log-mel spectrograms for Inkling dMel quantization.

Functions

Name	Description
`_to_exact_int`	Convert a floating sample count to an exact integer.
`_to_mono_audio`	Convert one audio clip to a mono fp32 waveform.

Data

LOGGER

__all__

API

class nemo_automodel.components.models.inkling.feature_extraction.InklingFeatureExtractor(
    feature_size: int = 80,
    sampling_rate: int = 16000,
    padding_value: float = 0.0,
    audio_token_duration_s: float = 0.05,
    window_size_multiplier: float = 2.0,
    n_fft: int | None = None,
    kwargs: typing.Any = {}
)

Bases: SequenceFeatureExtractor

Extract log-mel spectrograms for Inkling dMel quantization.

hop_length

mel_filters

model_input_names

= ['input_features', 'input_features_mask']

n_fft

= n_fft or self.window_size

window

window_size

nemo_automodel.components.models.inkling.feature_extraction.InklingFeatureExtractor.__call__(
    raw_speech: numpy.ndarray | torch.Tensor | list[float] | list[numpy.ndarray] | list[list[float]],
    sampling_rate: int | None = None,
    padding: bool | str | transformers.utils.PaddingStrategy = True,
    max_length: int | None = None,
    truncation: bool = False,
    pad_to_multiple_of: int | None = None,
    return_attention_mask: bool | None = True,
    return_tensors: str | transformers.utils.TensorType | None = None,
    device: str | torch.device = 'cpu',
    kwargs: typing.Any = {}
) -> transformers.feature_extraction_utils.BatchFeature

Extract log-mel features from one clip or a batch.

Parameters:

raw_speech

np.ndarray | torch.Tensor | list[float] | list[np.ndarray] | list[list[float]]

One waveform of shape [samples] or [samples, channels], or a list of such waveforms.

sampling_rate

int | NoneDefaults to None

Sampling rate used by the supplied waveform.

padding

bool | str | PaddingStrategyDefaults to True

Transformers padding strategy for the waveform batch.

max_length

int | NoneDefaults to None

Optional maximum waveform length in samples.

truncation

boolDefaults to False

Whether to truncate waveforms to max_length.

pad_to_multiple_of

int | NoneDefaults to None

Optional waveform padding multiple.

return_attention_mask

bool | NoneDefaults to True

Whether to return a valid-frame mask.

return_tensors

str | TensorType | NoneDefaults to None

Requested output tensor framework.

device

str | torch.deviceDefaults to 'cpu'

Device used for feature extraction.

**kwargs

AnyDefaults to {}

Additional padding arguments.

Returns: BatchFeature

A batch containing input_features with shape [batch, frames, mel_bins]

nemo_automodel.components.models.inkling.feature_extraction.InklingFeatureExtractor._extract_log_mel(
    waveform: torch.Tensor,
    device: torch.device
) -> torch.Tensor

Compute batched log-mel features.

Parameters:

waveform

torch.Tensor

Tensor of shape [batch, samples].

device

torch.device

Device used for STFT and filter-bank computation.

Returns: torch.Tensor

Tensor of shape [batch, frames, mel_bins].

nemo_automodel.components.models.inkling.feature_extraction._to_exact_int(
    value: float,
    name: str,
    tolerance: float = 1e-06
) -> int

Convert a floating sample count to an exact integer.

nemo_automodel.components.models.inkling.feature_extraction._to_mono_audio(
    clip: numpy.ndarray | torch.Tensor | list[float]
) -> torch.Tensor

Convert one audio clip to a mono fp32 waveform.

Parameters:

clip

np.ndarray | torch.Tensor | list[float]

Tensor or array of shape [samples] or [samples, channels].

Returns: torch.Tensor

Tensor of shape [samples] in fp32.

nemo_automodel.components.models.inkling.feature_extraction.LOGGER = logging.getLogger(__name__)

nemo_automodel.components.models.inkling.feature_extraction.__all__ = ['InklingFeatureExtractor']