Functions
Fixes parsing issues when reading from a file. |
|
|
Reads a file into a dataframe and performs any of the necessary cleanup. |
- cudf_json_onread_cleanup(x)[source]
Fixes parsing issues when reading from a file. When loading a JSON file, cuDF converts
\n
to\\n
for some reason.
- read_file_to_df(file_name, file_type, parser_kwargs={}, filter_nulls=True, df_type='pandas')[source]
Reads a file into a dataframe and performs any of the necessary cleanup.
- Parameters
- file_namestr
File to read.
- file_type<bsp-code-inline code="morpheus._lib.common.FileTypes">morpheus._lib.common.FileTypes</bsp-code-inline>
Type of file. Leave as Auto to determine from the extension.
- parser_kwargsdict, optional
Any argument to pass onto the parse, by default {}.
- filter_nullsbool, optional
Whether to filter null rows after loading, by default True.
- df_typetyping.Literal[, optional
What type of parser to use. Options are ‘cudf’ and ‘pandas’, by default “pandas”.
- Returns
- typing.Union[cudf.DataFrame, pandas.DataFrame]
A parsed DataFrame.