Custom functions called “Loaders” can be utilized by the DataLoader Module to load data into the pipeline. The user can choose to register their own customized loader function and add it to a dataloader registry, which will then become accessible to the DataLoader module during module loading.
Note : Loaders receive configuration from the
load task via control message during runtime.
- File to DataFrame Loader
- Filesystem Spec Loader
- SQL Loader