(Latest Version)

morpheus.modules.file_to_df

Functions

file_to_df(builder)

This module reads data from batched files into a DataFrame after receiving input from the "FileBatcher" module.

file_to_df(builder)[source]

This module reads data from batched files into a DataFrame after receiving input from the “FileBatcher” module. It can load file content from both local disk and S3 buckets.

Parameters
buildermrc.Builder

mrc Builder object.

Notes

Configurable parameters:
  • cache_dir (str): Directory to cache the rolling window data.

  • file_type (str): Type of the input file.

  • filter_null (bool): Whether to filter out null values.

  • parser_kwargs (dict): Keyword arguments to pass to the parser.

  • schema (dict): Schema of the input data.

  • timestamp_column_name (str): Name of the timestamp column.

© Copyright 2023, NVIDIA. Last updated on Apr 11, 2023.