nemo_curator.tasks.image

Module Contents

Classes

Name	Description
`ImageBatch`	Task for processing batches of images.
`ImageObject`	Represents a single image with metadata.
`ImageSampleTask`	Single-sample image task. `data` is an :class:`ImageTaskData` (or subclass).
`ImageTaskData`	Task data for image processing.

Data

T_ImageData

API

class nemo_curator.tasks.image.ImageBatch(
    dataset_name: str,
    data: list[nemo_curator.tasks.image.ImageObject] = list(),
    _stage_perf: list[nemo_curator.utils.performance_utils.StagePerfStats] = list(),
    _metadata: dict[str, typing.Any] = dict()
)

Dataclass

Bases: Task

Task for processing batches of images. Images are stored as a list of ImageObject instances, each containing the path to the image and associated metadata.

data

list[ImageObject] = field(default_factory=list)

num_items

int

Number of images in this batch.

nemo_curator.tasks.image.ImageBatch.validate() -> bool

Validate the task data.

class nemo_curator.tasks.image.ImageObject(
    image_path: str = '',
    image_id: str = '',
    metadata: dict[str, typing.Any] = dict(),
    image_data: numpy.ndarray | None = None,
    embedding: numpy.ndarray | None = None,
    aesthetic_score: float | None = None,
    nsfw_score: float | None = None
)

Dataclass

Represents a single image with metadata.

aesthetic_score

float | None = None

embedding

ndarray | None = None

image_data

ndarray | None = None

image_id

str = ''

image_path

str = ''

metadata

dict[str, Any] = field(default_factory=dict)

nsfw_score

float | None = None

class nemo_curator.tasks.image.ImageSampleTask(
    dataset_name: str,
    _stage_perf: list[nemo_curator.utils.performance_utils.StagePerfStats] = list(),
    _metadata: dict[str, typing.Any] = dict(),
    data: nemo_curator.tasks.image.T_ImageData
)

Dataclass

Bases: Task[T_ImageData]

Single-sample image task. data is an :class:ImageTaskData (or subclass).

data

T_ImageData

num_items

int

nemo_curator.tasks.image.ImageSampleTask.validate() -> bool

class nemo_curator.tasks.image.ImageTaskData(
    image_path: pathlib.Path | str,
    image_id: str | None = None,
    is_valid: bool = True,
    error: str | None = None
)

Dataclass

Task data for image processing.

error

str | None = None

image_id

str | None = None

image_path

Path | str

is_valid

bool = True

nemo_curator.tasks.image.ImageTaskData.to_dict() -> dict[str, typing.Any]

Convert to dictionary for JSON serialization.

Path objects are left as-is; callers should use default=str when passing to json.dumps to handle Path → str conversion.

nemo_curator.tasks.image.T_ImageData = TypeVar('T_ImageData', bound='ImageTaskData')