morpheus.loaders.fsspec_loader

Functions

fsspec_loader(control_message, task)

Loads data from external sources using the fsspec library, and returns an updated ControlMessage object with payload as MessageMeta, which contains a dataframe with file names and data.

fsspec_loader(control_message, task)[source]

Loads data from external sources using the fsspec library, and returns an updated ControlMessage object with payload as MessageMeta, which contains a dataframe with file names and data.

Parameters:
control_messageControlMessage

The ControlMessage object containing the pipeline control message.

tasktyping.Dict[any, any]

A dictionary representing the current task in the pipeline control message.

Returns:
control_messageControlMessage

An updated ControlMessage object with payload as a MessageMeta containing a dataframe with file names and data.

Raises:
RuntimeError:

If no files matched the input strings specified in the task, or if there was an error loading the data.

© Copyright 2023, NVIDIA. Last updated on Aug 23, 2023.