morpheus.io.data_record.DataRecord#

class DataRecord(
data_source,
data_label,
storage_type,
file_format='parquet',
copy_from_source=False,
)[source]#

Bases: object

Class for managing data records in different storage types and formats.

Attributes:

VALID_STORAGE_TYPES (tuple): Allowed storage types. VALID_FILE_FORMATS (tuple): Allowed file formats.

Attributes:
backing_source

Get the backing file for the data record.

data

Get the data associated with the data record.

data_label

Get the label for the data record.

format

Get the file format of the data record.

num_rows

Get the number of rows in the data record.

Methods

load()

Load a cuDF DataFrame from the DataRecord.
property backing_source: str#

Get the backing file for the data record.

Returns:

str: Backing file for the data record.

property data: cudf.DataFrame | pandas.DataFrame#

Get the data associated with the data record.

Returns:

Any: Data associated with the data record.

property data_label: str#

Get the label for the data record.

Returns:

str: Label for the data record.

property format: str#

Get the file format of the data record. Returns: str: File format of the data record.

load()[source]#

Load a cuDF DataFrame from the DataRecord.

Returns:

cudf.DataFrame: Loaded cuDF DataFrame.

property num_rows: int#

Get the number of rows in the data record.

Returns: int: Number of rows in the data record.