nemo_curator.models.audio.sed.cnn14

CNN14 decision-level model implementations for Sound Event Detection.

Vendored from the PANNs reference implementation at immutable commit d2f4b8c18eab44737fcc0de1248ae21eb43f6aa4 (MIT, copyright 2018-2020 Qiuqiang Kong), whose license is reproduced above. The model blocks and three decision-level CNN14 variants come from: https://github.com/qiuqiangkong/audioset_tagging_cnn/blob/d2f4b8c18eab44737fcc0de1248ae21eb43f6aa4/pytorch/models.py The interpolation and frame-padding helpers come from: https://github.com/qiuqiangkong/audioset_tagging_cnn/blob/d2f4b8c18eab44737fcc0de1248ae21eb43f6aa4/pytorch/pytorch_utils.py See Kong et al., “PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition” (2020).

Only the three decision-level variants are included because SED needs framewise output; the base Cnn14 emits clip-level output only. The code adds typing and factors shared helpers while retaining the reference forward call contract; the architecture and tensor semantics are unchanged, so published checkpoints load as-is.

Requires torchlibrosa, which arrives with the audio_cuda12 extra.

Module Contents

Classes

Name	Description
`AttBlock`	-
`Cnn14DecisionLevelAtt`	CNN14 with decision-level attention for SED.
`Cnn14DecisionLevelAvg`	CNN14 with decision-level average-pooling for SED.
`Cnn14DecisionLevelMax`	CNN14 with decision-level max-pooling for SED.
`ConvBlock`	-

Functions

Name	Description
`_cnn14_backbone`	Build the shared front-end layers for all CNN14 variants.
`_cnn14_encode`	Run shared CNN14 encoding to get feature maps. Returns (features, frames_num).
`init_bn`	Initialize a BatchNorm layer.
`init_layer`	Initialize a Linear or Conv layer.
`interpolate`	Interpolate in time to compensate CNN downsampling.
`pad_framewise_output`	Pad framewise output to match input frame count.

Data

msg

API

class nemo_curator.models.audio.sed.cnn14.AttBlock(
    n_in: int,
    n_out: int,
    activation: str = 'sigmoid'
)

Bases: Module

att

= nn.Conv1d(n_in, n_out, kernel_size=1, bias=True)

bn_att

= nn.BatchNorm1d(n_out)

cla

= nn.Conv1d(n_in, n_out, kernel_size=1, bias=True)

nemo_curator.models.audio.sed.cnn14.AttBlock._nonlinear(
    x: torch.Tensor
) -> torch.Tensor

nemo_curator.models.audio.sed.cnn14.AttBlock.forward(
    x: torch.Tensor
) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]

class nemo_curator.models.audio.sed.cnn14.Cnn14DecisionLevelAtt(
    sample_rate: int = 16000,
    window_size: int = 1024,
    hop_size: int = 320,
    mel_bins: int = 64,
    fmin: int = 50,
    fmax: int = 14000,
    classes_num: int = 527
)

Bases: Module

CNN14 with decision-level attention for SED.

att_block

= AttBlock(2048, classes_num, activation='sigmoid')

fc1

= nn.Linear(2048, 2048, bias=True)

interpolate_ratio

= 32

nemo_curator.models.audio.sed.cnn14.Cnn14DecisionLevelAtt._conv_blocks_list() -> list[nemo_curator.models.audio.sed.cnn14.ConvBlock]

nemo_curator.models.audio.sed.cnn14.Cnn14DecisionLevelAtt.forward(
    input: torch.Tensor,
    mixup_lambda: torch.Tensor | None = None
) -> dict[str, torch.Tensor]

class nemo_curator.models.audio.sed.cnn14.Cnn14DecisionLevelAvg(
    sample_rate: int = 16000,
    window_size: int = 1024,
    hop_size: int = 320,
    mel_bins: int = 64,
    fmin: int = 50,
    fmax: int = 14000,
    classes_num: int = 527
)

Bases: Module

CNN14 with decision-level average-pooling for SED.

fc1

= nn.Linear(2048, 2048, bias=True)

fc_audioset

= nn.Linear(2048, classes_num, bias=True)

interpolate_ratio

= 32

nemo_curator.models.audio.sed.cnn14.Cnn14DecisionLevelAvg._conv_blocks_list() -> list[nemo_curator.models.audio.sed.cnn14.ConvBlock]

nemo_curator.models.audio.sed.cnn14.Cnn14DecisionLevelAvg.forward(
    input: torch.Tensor,
    mixup_lambda: torch.Tensor | None = None
) -> dict[str, torch.Tensor]

class nemo_curator.models.audio.sed.cnn14.Cnn14DecisionLevelMax(
    sample_rate: int = 16000,
    window_size: int = 1024,
    hop_size: int = 320,
    mel_bins: int = 64,
    fmin: int = 50,
    fmax: int = 14000,
    classes_num: int = 527
)

Bases: Module

CNN14 with decision-level max-pooling for SED.

fc1

= nn.Linear(2048, 2048, bias=True)

fc_audioset

= nn.Linear(2048, classes_num, bias=True)

interpolate_ratio

= 32

nemo_curator.models.audio.sed.cnn14.Cnn14DecisionLevelMax._conv_blocks_list() -> list[nemo_curator.models.audio.sed.cnn14.ConvBlock]

nemo_curator.models.audio.sed.cnn14.Cnn14DecisionLevelMax.forward(
    input: torch.Tensor,
    mixup_lambda: torch.Tensor | None = None
) -> dict[str, torch.Tensor]

class nemo_curator.models.audio.sed.cnn14.ConvBlock(
    in_channels: int,
    out_channels: int
)

Bases: Module

bn1

= nn.BatchNorm2d(out_channels)

bn2

= nn.BatchNorm2d(out_channels)

conv1

conv2

nemo_curator.models.audio.sed.cnn14.ConvBlock.forward(
    x: torch.Tensor,
    pool_size: tuple[int, int] = (2, 2),
    pool_type: str = 'avg'
) -> torch.Tensor

nemo_curator.models.audio.sed.cnn14._cnn14_backbone(
    sample_rate: int,
    window_size: int,
    hop_size: int,
    mel_bins: int,
    fmin: int,
    fmax: int
) -> tuple[torchlibrosa.stft.Spectrogram, torchlibrosa.stft.LogmelFilterBank, torchlibrosa.augmentation.SpecAugmentation, torch.nn.BatchNorm2d, list[nemo_curator.models.audio.sed.cnn14.ConvBlock]]

Build the shared front-end layers for all CNN14 variants.

nemo_curator.models.audio.sed.cnn14._cnn14_encode(
    x: torch.Tensor,
    spec: torchlibrosa.stft.Spectrogram,
    logmel: torchlibrosa.stft.LogmelFilterBank,
    aug: torchlibrosa.augmentation.SpecAugmentation,
    bn0: torch.nn.BatchNorm2d,
    blocks: list[nemo_curator.models.audio.sed.cnn14.ConvBlock],
    training: bool
) -> tuple[torch.Tensor, int]

Run shared CNN14 encoding to get feature maps. Returns (features, frames_num).

nemo_curator.models.audio.sed.cnn14.init_bn(
    bn: torch.nn.BatchNorm1d | torch.nn.BatchNorm2d
) -> None

Initialize a BatchNorm layer.

nemo_curator.models.audio.sed.cnn14.init_layer(
    layer: torch.nn.Module
) -> None

Initialize a Linear or Conv layer.

nemo_curator.models.audio.sed.cnn14.interpolate(
    x: torch.Tensor,
    ratio: int
) -> torch.Tensor

Interpolate in time to compensate CNN downsampling.

Returns: (batch, time_steps * ratio, classes_num)

Parameters:

torch.Tensor

(batch, time_steps, classes_num)

ratio

int

upsample factor

nemo_curator.models.audio.sed.cnn14.pad_framewise_output(
    framewise_output: torch.Tensor,
    frames_num: int
) -> torch.Tensor

Pad framewise output to match input frame count.

nemo_curator.models.audio.sed.cnn14.msg = 'SED models require torchlibrosa. Install it with: pip install nemo-curator[audi...