morpheus.utils.nvt.schema_converters
Functions
<a href="#morpheus.utils.nvt.schema_converters.create_and_attach_nvt_workflow">create_and_attach_nvt_workflow</a>(input_schema) |
Converts an input_schema to a nvt.Workflow object. |
Classes
<a href="morpheus.utils.nvt.schema_converters.JSONFlattenInfo.html#morpheus.utils.nvt.schema_converters.JSONFlattenInfo">JSONFlattenInfo</a>(name, dtype, ...) |
Subclass of ColumnInfo. |
- create_and_attach_nvt_workflow(input_schema, visualize=False)[source]
Converts an
input_schemato anvt.Workflowobject.- Parameters
- input_schema
- visualize
Input schema which specifies how the DataFrame should be processed.
If True, the resulting workflow graph will be visualized. Default is False.
- Returns
- nvt.Workflow
A nvt.Workflow object representing the steps specified in the input schema.
- Raises
- ValueError
If the input schema is empty.
Notes
First we aggregate all preprocessing steps, which we assume are independent of each other and can be run in parallel.
Next we aggregate all column operations, which we assume are independent of each other and can be run in parallel and pass them the updated schema from the preprocessing steps.