NVIDIA Morpheus (24.06)
(Latest Version)

morpheus.loaders.fsspec_loader

Functions

fsspec_loader(control_message, task) Loads data from external sources using the fsspec library, and returns an updated ControlMessage object with payload as MessageMeta, which contains a dataframe with file names and data.
fsspec_loader(control_message, task)[source]

Loads data from external sources using the fsspec library, and returns an updated ControlMessage object with payload as MessageMeta, which contains a dataframe with file names and data.

Parameters
control_message

The ControlMessage object containing the pipeline control message.

task

A dictionary representing the current task in the pipeline control message.

Returns
control_message

An updated ControlMessage object with payload as a MessageMeta containing a dataframe with file names and data.

Raises
RuntimeError:

If no files matched the input strings specified in the task, or if there was an error loading the data.

Previous morpheus.loaders.file_to_df_loader
Next morpheus.loaders.sql_loader
© Copyright 2024, NVIDIA. Last updated on Jul 8, 2024.