nemo_rl.utils.path_utils

Path and IO helpers that also understand msc:// URLs; local paths are unchanged.

Module Contents

Functions

is_msc_path	Return whether a path explicitly selects an MSC storage profile.
maybe_enable_msc	Enable MSC in Megatron(-Bridge) for this process if any path is msc://.
checkpoint_abspath	Preserve MSC URLs when normalizing a checkpoint location.
save_checkpoint_object	Save controller checkpoint state beside local or MSC model shards.
load_checkpoint_object	Load controller checkpoint state saved by save_checkpoint_object.
path_exists	os.path.exists that also understands msc:// URLs.
open_path	open that also understands msc:// URLs.
as_path	Path(path), or the MSC path class for msc:// URLs.
path_isdir	os.path.isdir that also understands msc:// URLs.
listdir	os.listdir that also understands msc:// URLs.
rmtree	shutil.rmtree that also understands msc:// URLs.

API

nemo_rl.utils.path_utils.is_msc_path(path: str | os.PathLike[str]) → bool

Return whether a path explicitly selects an MSC storage profile.

nemo_rl.utils.path_utils.maybe_enable_msc(*paths: str | os.PathLike[str] | None) → None

Enable MSC in Megatron(-Bridge) for this process if any path is msc://.

Megatron(-Bridge) only treats msc:// as remote once its process-global MSC flag is on, so call this at every entry point that is handed a checkpoint path (load or save). Local-path runs never import Megatron.

nemo_rl.utils.path_utils.checkpoint_abspath(path: str | os.PathLike[str]) → str

Preserve MSC URLs when normalizing a checkpoint location.

nemo_rl.utils.path_utils.save_checkpoint_object(

value: Any,

path: str | os.PathLike[str],

) → None

Save controller checkpoint state beside local or MSC model shards.

nemo_rl.utils.path_utils.load_checkpoint_object(

path: str | os.PathLike[str],

**kwargs: Any,

) → Any

Load controller checkpoint state saved by save_checkpoint_object.

nemo_rl.utils.path_utils.path_exists(path: str | os.PathLike[str]) → bool

os.path.exists that also understands msc:// URLs.

nemo_rl.utils.path_utils.open_path(path: str | os.PathLike[str], mode: str = 'r') → Any

open that also understands msc:// URLs.

nemo_rl.utils.path_utils.as_path(path: str | os.PathLike[str]) → pathlib.Path

Path(path), or the MSC path class for msc:// URLs.

nemo_rl.utils.path_utils.path_isdir(path: str | os.PathLike[str], strict: bool = True) → bool

os.path.isdir that also understands msc:// URLs.

strict=False treats a missing MSC prefix as “not a directory” instead of raising.

nemo_rl.utils.path_utils.listdir(path: str | os.PathLike[str]) → list[str]

os.listdir that also understands msc:// URLs.

nemo_rl.utils.path_utils.rmtree(path: str | os.PathLike[str]) → None

shutil.rmtree that also understands msc:// URLs.