API ReferenceFull Library ReferenceNemo CuratorNemo CuratorTasks

nemo_curator.tasks.image

View as Markdown

Module Contents

Classes

NameDescription
ImageBatchTask for processing batches of images.
ImageObjectRepresents a single image with metadata.
ImageSampleTaskSingle-sample image task. data is an ImageTaskData (or subclass).
ImageTaskDataTask data for image processing.

Data

T_ImageData

API

class nemo_curator.tasks.ImageBatch(
    dataset_name: str,
    data: list[nemo_curator.tasks.image.ImageObject] = list(),
    _stage_perf: list[nemo_curator.utils.performance_utils.StagePerfStats] = list(),
    _metadata: dict[str, typing.Any] = dict()
)
Dataclass

Bases: Task

Task for processing batches of images. Images are stored as a list of ImageObject instances, each containing the path to the image and associated metadata.

data
list[ImageObject] = field(default_factory=list)
num_items
int

Number of images in this batch.

nemo_curator.tasks.ImageBatch.validate() -> bool

Validate the task data.

class nemo_curator.tasks.ImageObject(
    image_path: str = '',
    image_id: str = '',
    metadata: dict[str, typing.Any] = dict(),
    image_data: numpy.ndarray | None = None,
    embedding: numpy.ndarray | None = None,
    aesthetic_score: float | None = None,
    nsfw_score: float | None = None
)
Dataclass

Represents a single image with metadata.

aesthetic_score
float | None = None

Aesthetic quality score as float

embedding
ndarray | None = None

Image embedding vector as numpy array

image_data
ndarray | None = None

Raw image pixel data as numpy array (H, W, C) in RGB format

image_id
str = ''

Unique identifier for the image

image_path
str = ''

Path to the image file on disk

metadata
dict[str, Any] = field(default_factory=dict)

Additional metadata associated with the image

nsfw_score
float | None = None

NSFW probability score as float

class nemo_curator.tasks.image.ImageSampleTask(
    dataset_name: str,
    _stage_perf: list[nemo_curator.utils.performance_utils.StagePerfStats] = list(),
    _metadata: dict[str, typing.Any] = dict(),
    data: nemo_curator.tasks.image.T_ImageData
)
Dataclass

Bases: Task[T_ImageData]

Single-sample image task. data is an ImageTaskData (or subclass).

data
T_ImageData
num_items
int
nemo_curator.tasks.image.ImageSampleTask.validate() -> bool
class nemo_curator.tasks.image.ImageTaskData(
    image_path: pathlib.Path | str,
    image_id: str | None = None,
    is_valid: bool = True,
    error: str | None = None
)
Dataclass

Task data for image processing.

error
str | None = None
image_id
str | None = None
image_path
Path | str
is_valid
bool = True
nemo_curator.tasks.image.ImageTaskData.to_dict() -> dict[str, typing.Any]

Convert to dictionary for JSON serialization.

Path objects are left as-is; callers should use default=str when passing to json.dumps to handle Path → str conversion.

nemo_curator.tasks.image.T_ImageData = TypeVar('T_ImageData', bound='ImageTaskData')