nv_ingest.util.exception_handlers package#

Submodules#

nv_ingest.util.exception_handlers.converters module#

nv_ingest.util.exception_handlers.converters.datetools_exception_handler(
func: Callable,
**kwargs: Dict[str, Any],
) → Callable[source]#

A decorator that handles exceptions for date-related functions.

This decorator wraps a function that processes dates and catches any exceptions that occur during its execution. If an exception is raised, it logs a warning and returns the current UTC time as an ISO 8601 formatted string.

Parameters:

  • func (Callable) – The function to be decorated. This function is expected to handle date operations.

  • kwargs (dict) – Additional keyword arguments to be passed to the function.

Returns:

The wrapped function that executes func with exception handling.

Return type:

Callable

Notes

If an exception is raised while executing the wrapped function, the current UTC time (with timezone information removed) will be returned as an ISO 8601 formatted string.

Examples

>>> @datetools_exception_handler
... def parse_date(date_str):
...     return datetime.strptime(date_str, '%Y-%m-%d')
...
>>> parse_date('2024-08-22')
datetime.datetime(2024, 8, 22, 0, 0)

If the input is invalid, the current UTC time without timezone information is returned:

>>> parse_date('invalid-date')
'2024-08-22T12:34:56'
Raises:

Exception – Any exception raised by the wrapped function is caught, logged, and handled by returning the current UTC time.

nv_ingest.util.exception_handlers.decorators module#

class nv_ingest.util.exception_handlers.decorators.CMNVIngestFailureContextManager(
control_message: IngestControlMessage,
annotation_id: str,
raise_on_failure: bool = False,
func_name: str | None = None,
)[source]#

Bases: object

Context manager for handling IngestControlMessage failures during processing, providing a structured way to annotate and manage failures and successes.

Parameters:

  • control_message (IngestControlMessage) – The IngestControlMessage instance to be managed.

  • annotation_id (str) – The task’s unique identifier for annotation purposes.

  • raise_on_failure (bool, optional) – Determines whether to raise an exception upon failure. Defaults to False, which means failures are annotated rather than raising exceptions.

  • func_name (str, optional) – The name of the function being wrapped, used to annotate error messages uniformly. If None, stack introspection is used to deduce a likely function name. Defaults to None.

Return type:

None

nv_ingest.util.exception_handlers.decorators.cm_ensure_payload_not_null(
control_message: IngestControlMessage,
)[source]#

Ensures that the payload of a IngestControlMessage is not None.

Parameters:

control_message (IngestControlMessage) – The IngestControlMessage to check.

Raises:

ValueError – If the payload is None.

nv_ingest.util.exception_handlers.decorators.cm_set_failure(
control_message: IngestControlMessage,
reason: str,
) → IngestControlMessage[source]#

Sets the failure metadata on a IngestControlMessage.

Parameters:

  • control_message (IngestControlMessage) – The IngestControlMessage to set the failure metadata on.

  • reason (str) – The reason for the failure.

Returns:

control_message – The modified IngestControlMessage with the failure metadata set.

Return type:

IngestControlMessage

nv_ingest.util.exception_handlers.decorators.nv_ingest_node_failure_context_manager(
annotation_id: str,
payload_can_be_empty: bool = False,
raise_on_failure: bool = False,
skip_processing_if_failed: bool = True,
forward_func=None,
) → Callable[source]#

A decorator that applies a default failure context manager around a function to manage the execution and potential failure of operations involving IngestControlMessages.

Parameters:

  • annotation_id (str) – A unique identifier used for annotating the task’s result.

  • payload_can_be_empty (bool, optional) – If False, the payload of the IngestControlMessage will be checked to ensure it’s not null, raising an exception if it is null. Defaults to False, enforcing payload presence.

  • raise_on_failure (bool, optional) – If True, an exception is raised if the decorated function encounters an error. Otherwise, the error is handled silently by annotating the IngestControlMessage. Defaults to False.

  • skip_processing_if_failed (bool, optional) – If True, skips the processing of the decorated function if the control message has already been marked as failed. If False, the function will be processed regardless of the failure status of the IngestControlMessage. Defaults to True.

  • forward_func (callable, optional) – A function to forward the IngestControlMessage if it has already been marked as failed.

Returns:

A decorator that wraps the given function with failure handling logic.

Return type:

Callable

nv_ingest.util.exception_handlers.decorators.nv_ingest_source_failure_context_manager(
annotation_id: str,
payload_can_be_empty: bool = False,
raise_on_failure: bool = False,
) → Callable[source]#

A decorator that ensures any function’s output is treated as a IngestControlMessage for annotation. It applies a context manager to handle success and failure annotations based on the function’s execution.

Parameters:

  • annotation_id (str) – Unique identifier used for annotating the function’s output.

  • payload_can_be_empty (bool, optional) – Specifies if the function’s output IngestControlMessage payload can be empty, default is False.

  • raise_on_failure (bool, optional) – Determines if an exception should be raised upon function failure, default is False.

Returns:

A decorator that ensures function output is processed for success or failure annotation.

Return type:

Callable

nv_ingest.util.exception_handlers.detectors module#

nv_ingest.util.exception_handlers.detectors.langdetect_exception_handler(
func: Callable,
**kwargs: Dict[str, Any],
) → Callable[source]#

A decorator that handles LangDetectException for language detection functions.

This decorator wraps a function that performs language detection and catches any LangDetectException that occurs during its execution. If such an exception is raised, it logs a warning and returns a default value of LanguageEnum.UNKNOWN.

Parameters:

  • func (callable) – The function to be decorated. This function is expected to handle language detection.

  • kwargs (dict) – Additional keyword arguments to be passed to the function.

Returns:

The wrapped function that executes func with exception handling.

Return type:

callable

Notes

If a LangDetectException is raised while executing the wrapped function, the exception is logged, and LanguageEnum.UNKNOWN is returned as a fallback value.

Examples

>>> @langdetect_exception_handler
... def detect_language(text):
...     # Function implementation here
...     pass
...
>>> detect_language('This is a test sentence.')
<LanguageEnum.EN: 'en'>

If a LangDetectException is encountered, the function will return LanguageEnum.UNKNOWN:

>>> detect_language('')
<LanguageEnum.UNKNOWN: 'unknown'>
Raises:

LangDetectException – The exception raised by the wrapped function is caught and handled by logging a warning and returning LanguageEnum.UNKNOWN.

nv_ingest.util.exception_handlers.pdf module#

nv_ingest.util.exception_handlers.pdf.create_exception_tag(error_message, source_id=None)[source]#

Creates a metadata tag for logging or reporting an exception.

This function generates a metadata dictionary containing information about the exception, including the task type, status, source identifier, and error message. The metadata is validated and returned as a list containing a single entry.

Parameters:

  • error_message (str) – The error message describing the exception.

  • source_id (Optional[str], default=None) – The identifier for the source related to the error, if available.

Returns:

A list containing a single entry, which is a tuple. The first element of the tuple is None, and the second element is the validated metadata dictionary as a dict.

Return type:

list

Notes

This function is typically used to generate error metadata for tracking and logging purposes.

Examples

>>> create_exception_tag("File not found", source_id="12345")
[[None, {'task': 'EXTRACT', 'status': 'ERROR', 'source_id': '12345', 'error_msg': 'File not found'}]]
Raises:

ValidationError – If the metadata does not pass validation.

nv_ingest.util.exception_handlers.pdf.pdfium_exception_handler(descriptor)[source]#

A decorator that handles exceptions for functions interacting with PDFium.

This decorator wraps a function and catches any exceptions that occur during its execution. If an exception is raised, it logs a warning with a descriptor and the function name, then returns an empty list as a fallback value.

Parameters:

descriptor (str) – A string descriptor to identify the context or source of the function being wrapped. This descriptor is included in the log message if an exception occurs.

Returns:

A decorator function that wraps the target function with exception handling.

Return type:

callable

Notes

This decorator is useful for ensuring that functions interacting with PDFium can gracefully handle errors without interrupting the entire processing pipeline.

Examples

>>> @pdfium_exception_handler("PDF Processing")
... def process_pdf(file_path):
...     # Function implementation here
...     pass
...
>>> process_pdf("example.pdf")
[]
Raises:

Exception – Any exception raised by the wrapped function is caught, logged, and handled by returning an empty list.

nv_ingest.util.exception_handlers.schemas module#

nv_ingest.util.exception_handlers.schemas.schema_exception_handler(func, **kwargs)[source]#

A decorator that handles ValidationError exceptions for schema validation functions.

This decorator wraps a function that performs schema validation using Pydantic. If a ValidationError is raised, it logs detailed error messages and raises a ValueError with the combined error messages.

Parameters:

  • func (callable) – The function to be decorated. This function is expected to perform schema validation.

  • kwargs (dict) – Additional keyword arguments to be passed to the function.

Returns:

The wrapped function that executes func with exception handling.

Return type:

callable

Raises:

ValueError – If a ValidationError is caught, this decorator logs the error details and raises a ValueError with the combined error messages.

Notes

This decorator is particularly useful for functions that validate configurations or data models, ensuring that any validation errors are logged and communicated clearly.

Examples

>>> @schema_exception_handler
... def validate_config(config_data):
...     schema = MySchema(**config_data)
...     return schema
...
>>> try:
...     validate_config(invalid_config)
... except ValueError as e:
...     print(f"Caught error: {e}")
Caught error: Invalid configuration: field1: value is not a valid integer; field2: field required

Module contents#