core.dist_checkpointing.strategies.cached_metadata_filesystem_reader

FS Reader with metadata cached support.

Module Contents

Classes

CachedMetadataFileSystemReader

Extends FileSystemReader to cache metadata for improved performance.

API

class core.dist_checkpointing.strategies.cached_metadata_filesystem_reader.CachedMetadataFileSystemReader(path: Union[str, os.PathLike])

Bases: torch.distributed.checkpoint.FileSystemReader

Extends FileSystemReader to cache metadata for improved performance.

.. attribute:: _cached_metadata

Cached metadata from the file system.

Type:

Metadata or None

Initialization

Initialize with file system path.

Parameters:

path (Union[str, os.PathLike]) – Path to the checkpoint directory or file.

read_metadata() → torch.distributed.checkpoint.Metadata

Read metadata from file system, caching for subsequent calls.

Returns:

Checkpoint metadata.

Return type:

Metadata