stages.image.filters.nsfw_filter

Module Contents

Classes

ImageNSFWFilterStage

Stage for filtering out NSFW images using NSFWScorer model.

API

class stages.image.filters.nsfw_filter.ImageNSFWFilterStage

Bases: nemo_curator.stages.image.filters.base.BaseFilterStage

Stage for filtering out NSFW images using NSFWScorer model.

This class processes image batches through an NSFW scoring model to generate NSFW probability scores for each image. Images with scores above the threshold will be filtered out as NSFW content.

name: str

‘image_nsfw_filter’

process(

task: nemo_curator.tasks.ImageBatch,

) → nemo_curator.tasks.ImageBatch

Process an image batch to generate NSFW scores and filter by threshold.

Args: task: ImageBatch containing list of ImageObject instances with pre-computed embeddings

Returns: ImageBatch with filtered images that have NSFW scores below the threshold

setup(

_worker_metadata: nemo_curator.backends.base.WorkerMetadata | None = None,

) → None

Initialize the NSFW filtering model.

setup_on_node(

_node_info: nemo_curator.backends.base.NodeInfo | None = None,

_worker_metadata: nemo_curator.backends.base.WorkerMetadata | None = None,

) → None

Download NSFW model weights from LAION repository.

weights_path: str

None