(Latest Version)

morpheus.models.dfencoder.dataloader.DatasetFromDataframe

class DatasetFromDataframe(df, batch_size, preprocess_fn, shuffle_rows_in_batch=True, shuffle_batch_indices=False)[source]

Bases: torch.utils.data.dataset.Dataset

Attributes
num_samples

Returns the number of samples in the dataset.

Methods

convert_to_validation(model)

Converts the dataset to validation mode by resetting instance variables.

get_train_dataset(model, df)

A helper function to get a train dataset with the provided parameters.

get_validation_dataset(model, df)

A helper function to get a validation dataset with the provided parameters.

convert_to_validation(model)[source]

Converts the dataset to validation mode by resetting instance variables.

Parameters
modelAutoEncoder

The autoencoder model used to get relevant params and the preprocessing func.

static get_train_dataset(model, df)[source]

A helper function to get a train dataset with the provided parameters.

Parameters
modelAutoEncoder

The autoencoder model used to get relevant params and the preprocessing func.

dfpandas.DataFrame

Input dataframe used for the dataset.

Returns
DatasetFromDataframe

Training Dataset set up to load from the dataframe.

static get_validation_dataset(model, df)[source]

A helper function to get a validation dataset with the provided parameters.

Parameters
modelAutoEncoder

The autoencoder model used to get relevant params and the preprocessing func.

dfpandas.DataFrame

Input dataframe used for the dataset.

Returns
DatasetFromDataframe

Validation Dataset set up to load from the dataframe.

property num_samples

Returns the number of samples in the dataset.

© Copyright 2023, NVIDIA. Last updated on Apr 11, 2023.