morpheus.io.data_storage.file_system.FileSystemStorage
- class FileSystemStorage(file_path, file_format)[source]
Bases:
morpheus.io.data_storage_interface.RecordStorageInterfaceA class to manage storage of data in various file formats.
- Attributes
backing_sourceGet the backing source file path.
num_rowsGet the number of rows in the data.
ownerGet whether this instance is the owner of the backing source file.
Methods
- property backing_source: str
Get the backing source file path.
- Returns:
The path to the backing source file.
- delete()[source]
Delete the backing source file if this instance is the owner.
- load()[source]
Load data from the backing source file.
- Returns:
The loaded data as a cudf.DataFrame.
- property num_rows: int
Get the number of rows in the data.
- Returns:
The number of rows in the data.
- property owner: bool
Get whether this instance is the owner of the backing source file.
- Returns:
True if this instance is the owner of the backing source file, False otherwise.
- store(data_source, copy_from_source=False)[source]
Store data in the backing source file.
- Parameters
data_source (Union[pd.DataFrame, cudf.DataFrame, str]) – The data to store. Can be a pandas.DataFrame, cudf.DataFrame, or a file path.
copy_from_source (bool, optional) – If True, data_source is assumed to be a file path and the data will be copied from this file to the backing source file. If False, the backing source file will simply be updated to point to data_source (which is assumed to be a file path).