nemo_automodel.components.config.loader#

Module Contents#

Classes#

ConfigNode

A configuration node that wraps a dictionary (or parts of it) from a YAML file.

Functions#

set_enable_user_modules

Enable or disable loading user-defined code at runtime.

_is_safe_path

_is_allowed_module

Return True if a module is safe/allowed to import.

_is_safe_attr

_redact

translate_value

Convert a string token into the corresponding Python object.

load_module_from_file

Dynamically imports a module from a given file path.

_resolve_target

Resolve a dotted path to a Python object with safety checks.

load_yaml_config

Load a YAML configuration file and convert it to a ConfigNode.

Data#

ALLOWED_IMPORT_PREFIXES

SAFE_BASE_DIR

ENABLE_USER_MODULES

SENSITIVE_KEY_SUBSTRINGS

API#

nemo_automodel.components.config.loader.ALLOWED_IMPORT_PREFIXES#

(‘nemo_automodel’, ‘torch’, ‘transformers’, ‘torchdata’, ‘torchao’, ‘liger_kernel’)

nemo_automodel.components.config.loader.SAFE_BASE_DIR#

None

nemo_automodel.components.config.loader.ENABLE_USER_MODULES#

None

nemo_automodel.components.config.loader.SENSITIVE_KEY_SUBSTRINGS#

(‘password’, ‘secret’, ‘token’, ‘apikey’, ‘api_key’, ‘authorization’, ‘auth’)

nemo_automodel.components.config.loader.set_enable_user_modules(allow: bool) → None[source]#

Enable or disable loading user-defined code at runtime.

Users can also set environment variable NEMO_ENABLE_USER_MODULES=1 to enable.

nemo_automodel.components.config.loader._is_safe_path(p: pathlib.Path) → bool[source]#
nemo_automodel.components.config.loader._is_allowed_module(module_name: str) → bool[source]#

Return True if a module is safe/allowed to import.

Security policy (balanced for functionality and tests):

  • If user modules are explicitly enabled, allow everything.

  • Always allow modules that are already imported in this process.

  • Allow modules that are importable from the current PYTHONPATH/sys.path. This keeps behavior intuitive for local/test modules while still blocking truly unknown targets.

  • Fallback to explicit allowlist as a final gate (mostly relevant if find_spec returns None for the top-level name).

nemo_automodel.components.config.loader._is_safe_attr(name: str) → bool[source]#
nemo_automodel.components.config.loader._redact(obj: Any) → Any[source]#
nemo_automodel.components.config.loader.translate_value(v)[source]#

Convert a string token into the corresponding Python object.

This function first checks for a handful of special symbols (None/true/false), then falls back to ast.literal_eval, and finally to returning the original string if parsing fails.

Parameters:

v (str) – The raw string value to translate.

Returns:

  • None, True, or False for the special symbols

  • an int, float, tuple, list, dict, etc. if ast.literal_eval succeeds

  • the original string v if all parsing attempts fail

Return type:

The translated Python value, which may be

nemo_automodel.components.config.loader.load_module_from_file(file_path)[source]#

Dynamically imports a module from a given file path.

Intentionally permissive to support test/temporary modules. Caller is responsible for any higher-level policy checks.

nemo_automodel.components.config.loader._resolve_target(dotted_path: str)[source]#

Resolve a dotted path to a Python object with safety checks.

Supports two forms:

  • “path/to/file.py:attr” (file import): allowed if under SAFE_BASE_DIR unless opt-in is enabled.

  • “pkg.mod.attr” (dotted import): allowed only for allowlisted prefixes unless opt-in is enabled.

class nemo_automodel.components.config.loader.ConfigNode(d, raise_on_missing_attr=True)[source]#

A configuration node that wraps a dictionary (or parts of it) from a YAML file.

This class allows nested dictionaries and lists to be accessed as attributes and provides functionality to instantiate objects from configuration.

Initialization

Initialize the ConfigNode.

Parameters:

  • d (dict) – A dictionary representing configuration options.

  • raise_on_missing_attr (bool) – if True, it will return None on a missing attr.

__getattr__(key)[source]#
_wrap(k, v)[source]#

Wrap a configuration value based on its type.

Parameters:

  • k (str) – The key corresponding to the value.

  • v – The value to be wrapped.

Returns:

The wrapped value.

property raw_config#

Get the raw configuration dictionary.

Returns:

The raw configuration dictionary.

Return type:

dict

instantiate_path(dotted_path, default=None, *args, **kwargs)[source]#

Instantiate the target object specified in the configuration by path.

If the path is not found, returns the default value.

For example, this is useful when you want to do something like:

cfg_peft = self.cfg.get(“peft”, None) if cfg_peft is not None: cfg_peft = cfg_peft.instantiate()

In this case, you first check if the dotted path (in this case “peft”) is in the configuration. If it is, you instantiate it, otherwise you return the default value (in this case None).

With instantiate_path, you can do: cfg_peft = self.cfg.instantiate_path(“peft”, default=None)

Parameters:

  • dotted_path (str) – The path to the target object (e.g., “model.config”).

  • default – A default value to return if the path is not found.

  • *args – Positional arguments for the target instantiation.

  • **kwargs – Keyword arguments to override or add to the configuration values.

Returns:

The instantiated object.

instantiate(*args, **kwargs)[source]#

Instantiate the target object specified in the configuration.

This method looks for the “target” attribute in the configuration and resolves it to a callable function or class which is then instantiated.

Parameters:

  • *args – Positional arguments for the target instantiation.

  • **kwargs – Keyword arguments to override or add to the configuration values.

Returns:

The instantiated object.

Raises:

AttributeError – If no “target” attribute is found in the configuration.

_instantiate_value(v)[source]#

Recursively instantiate configuration values.

Parameters:

v – The configuration value.

Returns:

The instantiated value.

to_dict()[source]#

Convert the configuration node back to a dictionary.

Returns:

A dictionary representation of the configuration node.

Return type:

dict

_unwrap(v)[source]#

Recursively convert wrapped configuration values to basic Python types.

Parameters:

v – The configuration value.

Returns:

The unwrapped value.

get(key, default=None)[source]#

Retrieve a configuration value using a dotted key.

If any component of the path is missing, returns the specified default value.

Parameters:

  • key (str) – The dotted path key.

  • default – A default value to return if the key is not found.

Returns:

The configuration value or the default value.

set_by_dotted(dotted_key: str, value)[source]#

Set (or append) a value in the config using a dotted key.

e.g. set_by_dotted(“foo.bar.abc”, 1) will ensure self.foo.bar.abc == 1

__repr__(level=0)[source]#

Return a string representation of the configuration node with indentation.

Parameters:

level (int) – The current indentation level.

Returns:

An indented string representation of the configuration.

Return type:

str

_repr_value(value, level)[source]#

Format a configuration value for the string representation.

Parameters:

  • value – The configuration value.

  • level (int) – The indentation level.

Returns:

A formatted string representation of the value.

Return type:

str

__str__()[source]#

Return a string representation of the configuration node.

Returns:

The string representation.

Return type:

str

__contains__(key)[source]#

Check if a dotted key exists in the configuration.

Parameters:

key (str) – The dotted key to check.

Returns:

True if the key exists, False otherwise.

Return type:

bool

nemo_automodel.components.config.loader.load_yaml_config(path)[source]#

Load a YAML configuration file and convert it to a ConfigNode.

Parameters:

path (str) – The path to the YAML configuration file.

Returns:

A configuration node representing the YAML file.

Return type:

ConfigNode