morpheus.models.dfencoder.dataloader.DatasetFromDataframe
- class DatasetFromDataframe(df, batch_size, preprocess_fn, shuffle_rows_in_batch=True, shuffle_batch_indices=False)[source]
Bases:
torch.utils.data.dataset.Dataset- Attributes
num_samples
Returns the number of samples in the dataset.
Methods
convert_to_validation(model)Converts the dataset to validation mode by resetting instance variables.
get_train_dataset(model, df)A helper function to get a train dataset with the provided parameters.
get_validation_dataset(model, df)A helper function to get a validation dataset with the provided parameters.
- convert_to_validation(model)[source]
Converts the dataset to validation mode by resetting instance variables.
- Parameters
- modelAutoEncoder
The autoencoder model used to get relevant params and the preprocessing func.
- static get_train_dataset(model, df)[source]
A helper function to get a train dataset with the provided parameters.
- Parameters
- modelAutoEncoder
- dfpandas.DataFrame
The autoencoder model used to get relevant params and the preprocessing func.
Input dataframe used for the dataset.
- Returns
- DatasetFromDataframe
Training Dataset set up to load from the dataframe.
- static get_validation_dataset(model, df)[source]
A helper function to get a validation dataset with the provided parameters.
- Parameters
- modelAutoEncoder
- dfpandas.DataFrame
The autoencoder model used to get relevant params and the preprocessing func.
Input dataframe used for the dataset.
- Returns
- DatasetFromDataframe
Validation Dataset set up to load from the dataframe.
- property num_samples
Returns the number of samples in the dataset.