morpheus.loaders.fsspec_loader#
Functions
|
Loads data from external sources using the fsspec library, and returns an updated ControlMessage object with payload as MessageMeta, which contains a dataframe with file names and data. |
- fsspec_loader(control_message, task)[source]#
Loads data from external sources using the fsspec library, and returns an updated ControlMessage object with payload as MessageMeta, which contains a dataframe with file names and data.
- Parameters:
- control_messageControlMessage
The ControlMessage object containing the pipeline control message.
- tasktyping.Dict[any, any]
A dictionary representing the current task in the pipeline control message.
- Returns:
- control_messageControlMessage
An updated ControlMessage object with payload as a MessageMeta containing a dataframe with file names and data.
- Raises:
- RuntimeError:
If no files matched the input strings specified in the task, or if there was an error loading the data.