nvidia.dali.fn.io.file.read#

nvidia.dali.fn.io.file.read(
__filepaths,
/,
*,
bytes_per_sample_hint=[0],
dont_use_mmap=False,
preserve=False,
seed=-1,
use_o_direct=False,
device=None,
name=None,
)#

Reads raw file contents from an encoded filename represented by a 1D byte array.

Note

To produce a compatible encoded filepath from Python (e.g. in an external_source node generator), use np.frombuffer(filepath_str.encode(“utf-8”), dtype=types.UINT8).

Supported backends

  • ‘cpu’

Parameters:

__filepaths (TensorList) – File paths to read from.

Keyword Arguments:

  • bytes_per_sample_hint (int or list of int, optional, default = [0]) –

    Output size hint, in bytes per sample.

    If specified, the operator’s outputs residing in GPU or page-locked host memory will be preallocated to accommodate a batch of samples of this size.

  • dont_use_mmap (bool, optional, default = False) –

    If set to True, it will use plain file I/O instead of trying to map the file into memory.

    Mapping provides a small performance benefit when accessing a local file system, but for most network file systems, it does not provide a benefit

  • preserve (bool, optional, default = False) – Prevents the operator from being removed from the graph even if its outputs are not used.

  • seed (int, optional, default = -1) –

    Random seed.

    If not provided, it will be populated based on the global seed of the pipeline.

  • use_o_direct (bool, optional, default = False) –

    If set to True, the data will be read directly from the storage bypassing system cache.

    Mutually exclusive with dont_use_mmap=False.

On this page