nemo_automodel.shared.import_utils | NVIDIA NeMo AutoModel

Module Contents

Classes

Name	Description
`UnavailableError`	Error thrown if a symbol is unavailable due to an issue importing it.
`UnavailableMeta`	A metaclass for generating placeholder objects for unavailable symbols.

Functions

Name	Description
`get_check_model_inputs_decorator`	Get the appropriate check_model_inputs decorator based on transformers version.
`get_te_version`	Get TE version from version.
`get_torch_version`	Return pytorch version with fallback if unavailable.
`get_transformers_version`	Get transformers version from version.
`gpu_only_import`	A function used to import modules required only in GPU installs.
`gpu_only_import_from`	A function used to import symbols required only in GPU installs.
`is_te_min_version`	Check if minimum version of `transformer-engine` is installed.
`is_torch_min_version`	Check if minimum version of `torch` is installed.
`is_transformers_min_version`	Check if minimum version of `transformers` is installed.
`is_unavailable`	Helper to check if given symbol is actually a placeholder.
`null_decorator`	No-op decorator.
`safe_import`	A function used to import modules that may not be available.
`safe_import_from`	A function used to import symbols from modules that may not be available.
`safe_import_te`	Safely import Transformer Engine (TE). Returns (True, module) only if the

Data

GPU_INSTALL_STRING

MISSING_CUT_CROSS_ENTROPY_MSG

MISSING_QWEN_VL_UTILS_MSG

MISSING_TORCHAO_MSG

MISSING_TRITON_MSG

logger

API

class nemo_automodel.shared.import_utils.UnavailableError()

Exception

Bases: Exception

Error thrown if a symbol is unavailable due to an issue importing it.

class nemo_automodel.shared.import_utils.UnavailableMeta()

Bases: type

A metaclass for generating placeholder objects for unavailable symbols.

This metaclass allows errors to be deferred from import time to the time that a symbol is actually used in order to streamline the usage of optional dependencies. This is particularly useful for attempted imports of GPU-only modules which will only be invoked if GPU-only functionality is specifically used.

If an attempt to import a symbol fails, this metaclass is used to generate a class which stands in for that symbol. Any attempt to call the symbol (instantiate the class) or access its attributes will throw an UnavailableError exception. Furthermore, this class can be used in e.g. isinstance checks, since it will (correctly) fail to match any instance it is compared against.

In addition to calls and attribute access, a number of dunder methods are implemented so that other common usages of imported symbols (e.g. arithmetic) throw an UnavailableError, but this is not guaranteed for all possible uses. In such cases, other exception types (typically TypeErrors) will be thrown instead.

nemo_automodel.shared.import_utils.UnavailableMeta.__abs__()

nemo_automodel.shared.import_utils.UnavailableMeta.__add__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__call__(
    args = (),
    kwargs = {}
)

nemo_automodel.shared.import_utils.UnavailableMeta.__delete__(
    args = (),
    kwargs = {}
)

nemo_automodel.shared.import_utils.UnavailableMeta.__delitem__(
    name
)

nemo_automodel.shared.import_utils.UnavailableMeta.__divmod__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__enter__(
    args = (),
    kwargs = {}
)

nemo_automodel.shared.import_utils.UnavailableMeta.__eq__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__floordiv__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__ge__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__get__(
    args = (),
    kwargs = {}
)

nemo_automodel.shared.import_utils.UnavailableMeta.__getattr__(
    name
)

nemo_automodel.shared.import_utils.UnavailableMeta.__gt__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__hash__()

nemo_automodel.shared.import_utils.UnavailableMeta.__iadd__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__ifloordiv__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__ilshift__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__imul__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__index__()

nemo_automodel.shared.import_utils.UnavailableMeta.__invert__()

nemo_automodel.shared.import_utils.UnavailableMeta.__ipow__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__irshift__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__isub__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__iter__()

nemo_automodel.shared.import_utils.UnavailableMeta.__itruediv__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__le__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__len__()

nemo_automodel.shared.import_utils.UnavailableMeta.__lshift__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__lt__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__mul__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__ne__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__neg__()

nemo_automodel.shared.import_utils.UnavailableMeta.__new__(
    meta,
    name,
    bases,
    dct
)

nemo_automodel.shared.import_utils.UnavailableMeta.__pow__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__radd__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__rdivmod__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__rfloordiv__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__rlshift__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__rmul__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__rpow__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__rrshift__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__rshift__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__rsub__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__rtruediv__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__setitem__(
    name,
    value
)

nemo_automodel.shared.import_utils.UnavailableMeta.__sub__(
    other
)

nemo_automodel.shared.import_utils.UnavailableMeta.__truediv__(
    other
)

nemo_automodel.shared.import_utils.get_check_model_inputs_decorator()

Get the appropriate check_model_inputs decorator based on transformers version.

In transformers >= 4.57.3, check_model_inputs became a function that returns a decorator. In older versions, it was directly a decorator. In transformers >= 5.2.0, check_model_inputs was removed and split into merge_with_config_defaults and capture_outputs.

Returns:

Decorator function to validate model inputs.

nemo_automodel.shared.import_utils.get_te_version()

Get TE version from version.

nemo_automodel.shared.import_utils.get_torch_version()

Return pytorch version with fallback if unavailable.

Returns:

Pytorch’s version

nemo_automodel.shared.import_utils.get_transformers_version()

Get transformers version from version.

nemo_automodel.shared.import_utils.gpu_only_import(
    module,
    alt = None
)

A function used to import modules required only in GPU installs.

This function will attempt to import a module with the given name. This function will attempt to import a symbol with the given name from the given module, but it will not throw an ImportError if the symbol is not found. Instead, it will return a placeholder object which will raise an exception only if used with instructions on installing a GPU build.

Parameters:

module

str

The name of the module to import.

alt

objectDefaults to None

An optional module to be used in place of the given module if it fails to import in a non-GPU-enabled install

Returns:

The imported module, the given alternate, or a class derived from

nemo_automodel.shared.import_utils.gpu_only_import_from(
    module,
    symbol,
    alt = None
)

A function used to import symbols required only in GPU installs.

Parameters:

module

str

The name of the module to import.

symbol

str

The name of the symbol to import.

alt

objectDefaults to None

An optional object to be used in place of the given symbol if it fails to import in a non-GPU-enabled install

Returns:

The imported symbol, the given alternate, or a class derived from

nemo_automodel.shared.import_utils.is_te_min_version(
    version,
    check_equality = True
)

Check if minimum version of transformer-engine is installed.

nemo_automodel.shared.import_utils.is_torch_min_version(
    version,
    check_equality = True
)

Check if minimum version of torch is installed.

nemo_automodel.shared.import_utils.is_transformers_min_version(
    version,
    check_equality = True
)

Check if minimum version of transformers is installed.

nemo_automodel.shared.import_utils.is_unavailable(
    obj
)

Helper to check if given symbol is actually a placeholder.

nemo_automodel.shared.import_utils.null_decorator(
    args = (),
    kwargs = {}
)

No-op decorator.

nemo_automodel.shared.import_utils.safe_import(
    module,
    msg = None,
    alt = None
)

A function used to import modules that may not be available.

This function will attempt to import a module with the given name, but it will not throw an ImportError if the module is not found. Instead, it will return a placeholder object which will raise an exception only if used.

Parameters:

module

str

The name of the module to import.

msg

str or NoneDefaults to None

An optional error message to be displayed if this module is used after a failed import.

alt

objectDefaults to None

An optional module to be used in place of the given module if it fails to import

Returns: (bool, object)

The imported module, the given alternate, or a class derived from

nemo_automodel.shared.import_utils.safe_import_from(
    module,
    symbol,
    msg = None,
    alt = None,
    fallback_module = None
)

A function used to import symbols from modules that may not be available.

This function will attempt to import a symbol with the given name from the given module, but it will not throw an ImportError if the symbol is not found. Instead, it will return a placeholder object which will raise an exception only if used.

Parameters:

module

str

The name of the module in which the symbol is defined.

symbol

str

The name of the symbol to import.

msg

str or NoneDefaults to None

An optional error message to be displayed if this symbol is used after a failed import.

alt

objectDefaults to None

An optional object to be used in place of the given symbol if it fails to import

fallback_module

strDefaults to None

Alternative name of the model in which the symbol is defined. The function will first to import using the module value and if that fails will also try the fallback_module.

Returns: (object, bool)

The imported symbol, the given alternate, or a class derived from

nemo_automodel.shared.import_utils.safe_import_te()

Safely import Transformer Engine (TE). Returns (True, module) only if the package and its PyTorch extension load successfully. Handles known failure modes so callers get a consistent (False, placeholder) instead of crashes.

Known failure modes (handled by returning False and a placeholder):

ImportError when importing transformer_engine (torch is pulled in by TE’s pytorch/init.py), e.g. CUDA/runtime mismatch: ImportError: …/libc10_cuda.so: undefined symbol: cudaGetDriverEntryPointByVersion, version libcudart.so.12
FileNotFoundError when TE loads its framework extension (e.g. in transformer_engine.common.load_framework_extension(“torch”)): FileNotFoundError: Could not find shared object file for Transformer Engine torch lib.
OSError when TE’s core .so is dlopen’d against an incompatible CUDA lib, e.g. a prebuilt wheel needing a newer cuBLAS than is installed: OSError: …/libtransformer_engine.so: undefined symbol: cublasLtGroupedMatrixLayoutInit_internal, version libcublasLt.so.13

Returns:

Tuple[bool, Union[module, UnavailableMeta]]: (True, te_module) if import

nemo_automodel.shared.import_utils.GPU_INSTALL_STRING = 'Install GPU packages via `pip install --extra-index-url https://pypi.nvidia.com...

nemo_automodel.shared.import_utils.MISSING_CUT_CROSS_ENTROPY_MSG = 'cut_cross_entropy is not installed. Please install it with `pip install cut-cro...

nemo_automodel.shared.import_utils.MISSING_QWEN_VL_UTILS_MSG = 'qwen_vl_utils is not installed. Please install it with `pip install qwen-vl-uti...

nemo_automodel.shared.import_utils.MISSING_TORCHAO_MSG = 'torchao is not installed. Please install it with `pip install torchao`.'

nemo_automodel.shared.import_utils.MISSING_TRITON_MSG = 'triton is not installed. Please install it with `pip install triton`.'

nemo_automodel.shared.import_utils.logger = logging.getLogger(__name__)