morpheus.io.deserializers

(Latest Version)

Functions

cudf_json_onread_cleanup(x)

Fixes parsing issues when reading from a file.

read_file_to_df(file_name[, file_type, ...])

Reads a file into a dataframe and performs any of the necessary cleanup.

cudf_json_onread_cleanup(x)[source]

Fixes parsing issues when reading from a file. When loading a JSON file, cuDF converts \n to \\n for some reason.

read_file_to_df(file_name, file_type=<FileTypes.Auto: 0>, parser_kwargs=None, filter_nulls=True, df_type='pandas')[source]

Reads a file into a dataframe and performs any of the necessary cleanup.

Parameters
file_namestr

File to read.

file_typemorpheus.common.FileTypes

Type of file. Leave as Auto to determine from the extension.

parser_kwargsdict, optional

Any argument to pass onto the parse, by default {}. Ignored when C++ execution is enabled and df_type="cudf"

filter_nullsbool, optional

Whether to filter null rows after loading, by default True.

df_typetyping.Literal[, optional

What type of parser to use. Options are ‘cudf’ and ‘pandas’, by default “pandas”.

Returns
typing.Union[cudf.DataFrame, pandas.DataFrame]

A parsed DataFrame.

© Copyright 2023, NVIDIA. Last updated on Apr 11, 2023.