`nemo_automodel.components.config.loader`#

Module Contents#

Classes#

`_OrigValueStr`	String that keeps its original (unresolved) value for safe printing.
`ConfigNode`	A configuration node that wraps a dictionary (or parts of it) from a YAML file.

Functions#

`set_enable_user_modules`	Enable or disable loading user-defined code at runtime.
`_is_safe_path`
`_is_allowed_module`	Return True if a module is safe/allowed to import.
`_is_safe_attr`
`_redact`
`translate_value`	Convert a string token into the corresponding Python object.
`resolve_yaml_env_vars`	Resolve env var references inside a YAML-loaded container.
`load_module_from_file`	Dynamically imports a module from a given file path.
`_resolve_target`	Resolve a dotted path to a Python object with safety checks.
`config_to_yaml_str`	Convert a config object to a YAML string suitable for logging/printing.
`load_yaml_config`	Load a YAML configuration file and convert it to a ConfigNode.

Data#

`ALLOWED_IMPORT_PREFIXES`
`SAFE_BASE_DIR`
`ENABLE_USER_MODULES`
`SENSITIVE_KEY_SUBSTRINGS`

API#

nemo_automodel.components.config.loader.ALLOWED_IMPORT_PREFIXES#: (‘nemo_automodel’, ‘torch’, ‘transformers’, ‘torchdata’, ‘torchao’, ‘liger_kernel’)

nemo_automodel.components.config.loader.SAFE_BASE_DIR#: None

nemo_automodel.components.config.loader.ENABLE_USER_MODULES#: None

nemo_automodel.components.config.loader.SENSITIVE_KEY_SUBSTRINGS#: (‘password’, ‘secret’, ‘token’, ‘apikey’, ‘api_key’, ‘authorization’, ‘auth’)

nemo_automodel.components.config.loader.set_enable_user_modules(allow: bool) → None[source]#

Enable or disable loading user-defined code at runtime.

Users can also set environment variable NEMO_ENABLE_USER_MODULES=1 to enable.

nemo_automodel.components.config.loader._is_safe_path(p: pathlib.Path) → bool[source]#

nemo_automodel.components.config.loader._is_allowed_module(module_name: str) → bool[source]#

Return True if a module is safe/allowed to import.

Security policy (balanced for functionality and tests):

If user modules are explicitly enabled, allow everything.
Always allow modules that are already imported in this process.
Allow modules that are importable from the current PYTHONPATH/sys.path. This keeps behavior intuitive for local/test modules while still blocking truly unknown targets.
Fallback to explicit allowlist as a final gate (mostly relevant if find_spec returns None for the top-level name).

nemo_automodel.components.config.loader._is_safe_attr(name: str) → bool[source]#

nemo_automodel.components.config.loader._redact(obj: Any) → Any[source]#

nemo_automodel.components.config.loader.translate_value(v)[source]#

Convert a string token into the corresponding Python object.

This function first checks for a handful of special symbols (None/true/false), then falls back to ast.literal_eval, and finally to returning the original string if parsing fails.

Parameters:

v (str) – The raw string value to translate.

Returns:

None, True, or False for the special symbols
an int, float, tuple, list, dict, etc. if ast.literal_eval succeeds
the original string v if all parsing attempts fail

Return type:

The translated Python value, which may be

class nemo_automodel.components.config.loader._OrigValueStr[source]#

Bases: str

String that keeps its original (unresolved) value for safe printing.

Initialization

Initialize self. See help(type(self)) for accurate signature.

__new__(value: str, orig_value: str)[source]#

nemo_automodel.components.config.loader.resolve_yaml_env_vars(obj: Any) → Any[source]#

Resolve env var references inside a YAML-loaded container.

Supported forms inside strings:

${VAR} / ${VAR,default}
${var.dot.var} (dots are treated as part of the env var name)
$VAR / $var.dot.var
Back-compat: ${oc.env:VAR} / ${oc.env:VAR,default}

nemo_automodel.components.config.loader.load_module_from_file(file_path)[source]#

Dynamically imports a module from a given file path.

Intentionally permissive to support test/temporary modules. Caller is responsible for any higher-level policy checks.

nemo_automodel.components.config.loader._resolve_target(dotted_path: str)[source]#

Resolve a dotted path to a Python object with safety checks.

Supports two forms:

“path/to/file.py:attr” (file import): allowed if under SAFE_BASE_DIR unless opt-in is enabled.
“pkg.mod.attr” (dotted import): allowed only for allowlisted prefixes unless opt-in is enabled.

class nemo_automodel.components.config.loader.ConfigNode(d, raise_on_missing_attr=True)[source]#

A configuration node that wraps a dictionary (or parts of it) from a YAML file.

This class allows nested dictionaries and lists to be accessed as attributes and provides functionality to instantiate objects from configuration.

Initialization

Initialize the ConfigNode.

Parameters:

d (dict) – A dictionary representing configuration options.
raise_on_missing_attr (bool) – if True, it will return None on a missing attr.

__getattr__(key)[source]#

_wrap(k, v)[source]#

Wrap a configuration value based on its type.

Parameters:

k (str) – The key corresponding to the value.
v – The value to be wrapped.

Returns:

The wrapped value.

property raw_config#

Get the raw configuration dictionary.

Returns:: The raw configuration dictionary.
Return type:: dict

instantiate_path(dotted_path, default=None, *args, **kwargs)[source]#

Instantiate the target object specified in the configuration by path.

If the path is not found, returns the default value.

For example, this is useful when you want to do something like:

cfg_peft = self.cfg.get(“peft”, None) if cfg_peft is not None: cfg_peft = cfg_peft.instantiate()

In this case, you first check if the dotted path (in this case “peft”) is in the configuration. If it is, you instantiate it, otherwise you return the default value (in this case None).

With instantiate_path, you can do: cfg_peft = self.cfg.instantiate_path(“peft”, default=None)

Parameters:

dotted_path (str) – The path to the target object (e.g., “model.config”).
default – A default value to return if the path is not found.
*args – Positional arguments for the target instantiation.
**kwargs – Keyword arguments to override or add to the configuration values.

Returns:

The instantiated object.

instantiate(*args, **kwargs)[source]#

Instantiate the target object specified in the configuration.

This method looks for the “target” attribute in the configuration and resolves it to a callable function or class which is then instantiated.

Parameters:

*args – Positional arguments for the target instantiation.
**kwargs – Keyword arguments to override or add to the configuration values.

Returns:

The instantiated object.

Raises:

AttributeError – If no “target” attribute is found in the configuration.

_instantiate_value(v)[source]#

Recursively instantiate configuration values.

Parameters:: v – The configuration value.
Returns:: The instantiated value.

to_dict()[source]#

Convert the configuration node back to a dictionary.

Returns:: A dictionary representation of the configuration node.
Return type:: dict

_to_dotted_path(obj)[source]#

Convert a callable/class/method object to a dotted path string.

Best-effort normalization for a few common cases to produce concise, user-friendly paths.

to_yaml_dict( *, resolve_env: bool = False, redact_sensitive: bool = False, use_orig_values: bool = False, )[source]#

Convert configuration to a YAML-ready dictionary:

Preserves typed scalars (ints, floats, bools)
Converts callables/classes/methods (e.g., target, *_fn) to dotted path strings
Recurses through nested ConfigNodes and lists

Parameters:

resolve_env – If True, resolve ${oc.env:VAR} interpolations in the returned dict. This does not mutate the in-memory config.
redact_sensitive – If True, redact values for keys that look sensitive (token/secret/etc).
use_orig_values – If True, prefer _orig_value (when present) for safe printing/logging.

_unwrap(v)[source]#

Recursively convert wrapped configuration values to basic Python types.

Parameters:: v – The configuration value.
Returns:: The unwrapped value.

get_as_string(key, default=None)[source]#

Get the string representation of a configuration value.

If the value is a function or class (resolved from an import path), returns the original import path string. Otherwise returns the value as a string.

Parameters:: key (str) – The key to look up.
Returns:: The string representation of the value, or None if key not found.
Return type:: str

get(key, default=None)[source]#

Retrieve a configuration value using a dotted key.

If any component of the path is missing, returns the specified default value.

Parameters:

key (str) – The dotted path key.
default – A default value to return if the key is not found.

Returns:

The configuration value or the default value.

set_by_dotted(dotted_key: str, value)[source]#

Set (or append) a value in the config using a dotted key.

e.g. set_by_dotted(“foo.bar.abc”, 1) will ensure self.foo.bar.abc == 1

__repr__(level: int = 0, *, use_orig_values: bool = True)[source]#

Return a string representation of the configuration node with indentation.

Parameters:

level (int) – The current indentation level.
use_orig_values (bool) – If True, prefer original placeholder strings (e.g. ${VAR}) stored on _OrigValueStr values for safe logging. If False, show resolved values.

Returns:

An indented string representation of the configuration.

Return type:

str

_repr_value(value, level, *, use_orig_values: bool = True)[source]#

Format a configuration value for the string representation.

Parameters:

value – The configuration value.
level (int) – The indentation level.
use_orig_values (bool) – If True, prefer original placeholder strings stored on _OrigValueStr values for safe logging. If False, show resolved values.

Returns:

A formatted string representation of the value.

Return type:

str

__str__()[source]#

Return a string representation of the configuration node.

Returns:: The string representation.
Return type:: str

__contains__(key)[source]#

Check if a dotted key exists in the configuration.

Parameters:: key (str) – The dotted key to check.
Returns:: True if the key exists, False otherwise.
Return type:: bool

nemo_automodel.components.config.loader.config_to_yaml_str(cfg_obj, *, use_orig_values: bool = True)[source]#

Convert a config object to a YAML string suitable for logging/printing.

Uses original placeholder strings (e.g. ${VAR}) and original target/*_fn strings when use_orig_values is True; never includes internal keys like _original_strings. If cfg_obj is a ConfigNode, uses to_yaml_dict(); otherwise falls back to to_dict() or a plain dict.

nemo_automodel.components.config.loader.load_yaml_config(path)[source]#

Load a YAML configuration file and convert it to a ConfigNode.

Parameters:: path (str) – The path to the YAML configuration file.
Returns:: A configuration node representing the YAML file.
Return type:: ConfigNode

nemo_automodel.components.config.loader#

Module Contents#

Classes#

Functions#

Data#

API#

`nemo_automodel.components.config.loader`#