Loaders

Custom functions called “Loaders” can be utilized by the DataLoader Module to load data into the pipeline. The user can choose to register their own customized loader function and add it to a dataloader registry, which will then become accessible to the DataLoader module during module loading.

Note : Loaders receive configuration from the load task via control message during runtime.

• File to DataFrame Loader
  
  • Example Loader Configuration
  • Task Configurable Parameters
  • batcher_config
  • Example Load Task Configuration
• Filesystem Spec Loader
  
  • Example Loader Configuration
  • Task Configurable Parameters
  • Example Load Task Configuration
• SQL Loader
  
  • Example Loader Configuration
  • Task Configurable Parameters
  • Example Load Task Configuration
• REST to DataFrame Loader
  
  • Example Loader Configuration
  • Task Configurable Parameters
  • queries
  • Example Load Task Configuration