Data Loader

Module: polygraphy.comparator

class DataLoader(seed=None, iterations=None, input_metadata=None, int_range=None, float_range=None, val_range=None, data_loader_backend_module=None)[source]

Bases: object

Generates synthetic input data.

Parameters:

  • seed (int) – The seed to use when generating random inputs. Defaults to util.constants.DEFAULT_SEED.

  • iterations (int) – The number of iterations for which to supply data. Defaults to 1.

  • input_metadata (TensorMetadata) – A mapping of input names to their corresponding shapes and data types. This will be used to determine what shapes to supply for inputs with dynamic shape, as well as to set the data type of the generated inputs. If either dtype or shape are None, then the value will be automatically determined. For input shape tensors, i.e. inputs whose value describes a shape in the model, the provided shape will be used to populate the values of the inputs, rather than to determine their shape.

  • val_range (Union[Tuple[number], Dict[str, Tuple[number]]]) – A tuple containing exactly 2 numbers, indicating the minimum and maximum values (inclusive) the data loader should generate. If either value in the tuple is None, the default will be used for that value. If None is provided instead of a tuple, then the default values will be used for both the minimum and maximum. This can be specified on a per-input basis using a dictionary. In that case, use an empty string (“”) as the key to specify default range for inputs not explicitly listed. Defaults to (0.0, 1.0).

  • data_loader_backend_module (str) – A string denoting what module to use to construct the input data arrays. Currently supports “numpy” and “torch”. Defaults to “numpy”.

  • int_range (Tuple[int]) – [DEPRECATED - Use val_range instead] A tuple containing exactly 2 integers, indicating the minimum and maximum integer values (inclusive) the data loader should generate. If either value in the tuple is None, the default will be used for that value. If None is provided instead of a tuple, then the default values will be used for both the minimum and maximum.

  • float_range (Tuple[float]) – [DEPRECATED - Use val_range instead] A tuple containing exactly 2 floats, indicating the minimum and maximum float values (inclusive) the data loader should generate. If either value in the tuple is None, the default will be used for that value. If None is provided instead of a tuple, then the default values will be used for both the minimum and maximum.

__getitem__(index)[source]

Generates random input data.

May update the DataLoader’s input_metadata attribute.

Parameters:

index (int) – Since this class behaves like an iterable, it takes an index parameter. Generated data is guaranteed to be the same for the same index.

Returns:

A mapping of input names to input numpy buffers.

Return type:

OrderedDict[str, Union[numpy.ndarray, torch.Tensor]]