nv_ingest_api.util.exception_handlers package#

Submodules#

nv_ingest_api.util.exception_handlers.converters module#

nv_ingest_api.util.exception_handlers.converters.datetools_exception_handler(
func: Callable,
**kwargs: Dict[str, Any],
) → Callable[source]#

A decorator that handles exceptions for date-related functions.

This decorator wraps a function that processes dates and catches any exceptions that occur during its execution. If an exception is raised, it logs a warning and returns the current UTC time as an ISO 8601 formatted string.

Parameters:

  • func (Callable) – The function to be decorated. This function is expected to handle date operations.

  • kwargs (dict) – Additional keyword arguments to be passed to the function.

Returns:

The wrapped function that executes func with exception handling.

Return type:

Callable

Notes

If an exception is raised while executing the wrapped function, the current UTC time (with timezone information removed) will be returned as an ISO 8601 formatted string.

Examples

>>> @datetools_exception_handler
... def parse_date(date_str):
...     return datetime.strptime(date_str, '%Y-%m-%d')
...
>>> parse_date('2024-08-22')
datetime.datetime(2024, 8, 22, 0, 0)

If the input is invalid, the current UTC time without timezone information is returned:

>>> parse_date('invalid-date')
'2024-08-22T12:34:56'
Raises:

Exception – Any exception raised by the wrapped function is caught, logged, and handled by returning the current UTC time.

nv_ingest_api.util.exception_handlers.decorators module#

class nv_ingest_api.util.exception_handlers.decorators.CMNVIngestFailureContextManager(
control_message: IngestControlMessage,
annotation_id: str,
raise_on_failure: bool = False,
func_name: str | None = None,
)[source]#

Bases: object

Context manager for handling IngestControlMessage failures during processing, providing a structured way to annotate and manage failures and successes.

Parameters:

  • control_message (IngestControlMessage) – The IngestControlMessage instance to be managed.

  • annotation_id (str) – The task’s unique identifier for annotation purposes.

  • raise_on_failure (bool, optional) – Determines whether to raise an exception upon failure. Defaults to False, which means failures are annotated rather than raising exceptions.

  • func_name (str, optional) – The name of the function being wrapped, used to annotate error messages uniformly. If None, stack introspection is used to deduce a likely function name. Defaults to None.

Return type:

None

nv_ingest_api.util.exception_handlers.decorators.nv_ingest_node_failure_context_manager(
annotation_id: str,
payload_can_be_empty: bool = False,
raise_on_failure: bool = False,
skip_processing_if_failed: bool = True,
forward_func: Callable[[Any], Any] | None = None,
) → Callable[source]#

Decorator that applies a failure context manager around a function processing an IngestControlMessage. Works with both synchronous and asynchronous functions, and supports class methods (with ‘self’).

Parameters:

  • annotation_id (str) – A unique identifier for annotation.

  • payload_can_be_empty (bool, optional) – If False, the message payload must not be null.

  • raise_on_failure (bool, optional) – If True, exceptions are raised; otherwise, they are annotated.

  • skip_processing_if_failed (bool, optional) – If True, skip processing if the message is already marked as failed.

  • forward_func (Optional[Callable[[Any], Any]]) – If provided, a function to forward the message when processing is skipped.

Returns:

The decorated function.

Return type:

Callable

nv_ingest_api.util.exception_handlers.decorators.nv_ingest_node_failure_try_except(
annotation_id: str,
payload_can_be_empty: bool = False,
raise_on_failure: bool = False,
skip_processing_if_failed: bool = True,
forward_func: Callable[[Any], Any] | None = None,
) → Callable[source]#

Decorator that wraps function execution in a try/except block to handle failures by annotating an IngestControlMessage. Replaces the context manager approach for potentially simpler interaction with frameworks like Ray.

Parameters are the same as nv_ingest_node_failure_context_manager.

nv_ingest_api.util.exception_handlers.decorators.nv_ingest_source_failure_context_manager(
annotation_id: str,
payload_can_be_empty: bool = False,
raise_on_failure: bool = False,
) → Callable[source]#

A decorator that ensures any function’s output is treated as a IngestControlMessage for annotation. It applies a context manager to handle success and failure annotations based on the function’s execution.

Parameters:

  • annotation_id (str) – Unique identifier used for annotating the function’s output.

  • payload_can_be_empty (bool, optional) – Specifies if the function’s output IngestControlMessage payload can be empty, default is False.

  • raise_on_failure (bool, optional) – Determines if an exception should be raised upon function failure, default is False.

Returns:

A decorator that ensures function output is processed for success or failure annotation.

Return type:

Callable

nv_ingest_api.util.exception_handlers.decorators.unified_exception_handler(func)[source]#

nv_ingest_api.util.exception_handlers.detectors module#

nv_ingest_api.util.exception_handlers.detectors.langdetect_exception_handler(
func: Callable,
**kwargs: Dict[str, Any],
) → Callable[source]#

A decorator that handles LangDetectException for language detection functions.

This decorator wraps a function that performs language detection and catches any LangDetectException that occurs during its execution. If such an exception is raised, it logs a warning and returns a default value of LanguageEnum.UNKNOWN.

Parameters:

  • func (callable) – The function to be decorated. This function is expected to handle language detection.

  • kwargs (dict) – Additional keyword arguments to be passed to the function.

Returns:

The wrapped function that executes func with exception handling.

Return type:

callable

Notes

If a LangDetectException is raised while executing the wrapped function, the exception is logged, and LanguageEnum.UNKNOWN is returned as a fallback value.

Examples

>>> @langdetect_exception_handler
... def detect_language(text):
...     # Function implementation here
...     pass
...
>>> detect_language('This is a test sentence.')
<LanguageEnum.EN: 'en'>

If a LangDetectException is encountered, the function will return LanguageEnum.UNKNOWN:

>>> detect_language('')
<LanguageEnum.UNKNOWN: 'unknown'>
Raises:

LangDetectException – The exception raised by the wrapped function is caught and handled by logging a warning and returning LanguageEnum.UNKNOWN.

nv_ingest_api.util.exception_handlers.pdf module#

nv_ingest_api.util.exception_handlers.pdf.create_exception_tag(error_message, source_id=None)[source]#

Creates a metadata tag for logging or reporting an exception.

This function generates a metadata dictionary containing information about the exception, including the task type, status, source identifier, and error message. The metadata is validated and returned as a list containing a single entry.

Parameters:

  • error_message (str) – The error message describing the exception.

  • source_id (Optional[str], default=None) – The identifier for the source related to the error, if available.

Returns:

A list containing a single entry, which is a tuple. The first element of the tuple is None, and the second element is the validated metadata dictionary as a dict.

Return type:

list

Notes

This function is typically used to generate error metadata for tracking and logging purposes.

Examples

>>> create_exception_tag("File not found", source_id="12345")
[[None, {'task': 'EXTRACT', 'status': 'ERROR', 'source_id': '12345', 'error_msg': 'File not found'}]]
Raises:

ValidationError – If the metadata does not pass validation.

nv_ingest_api.util.exception_handlers.pdf.pdfium_exception_handler(descriptor)[source]#

A decorator that handles exceptions for functions interacting with PDFium.

This decorator wraps a function and catches any exceptions that occur during its execution. If an exception is raised, it logs a warning with a descriptor and the function name, then returns an empty list as a fallback value.

Parameters:

descriptor (str) – A string descriptor to identify the context or source of the function being wrapped. This descriptor is included in the log message if an exception occurs.

Returns:

A decorator function that wraps the target function with exception handling.

Return type:

callable

Notes

This decorator is useful for ensuring that functions interacting with PDFium can gracefully handle errors without interrupting the entire processing pipeline.

Examples

>>> @pdfium_exception_handler("PDF Processing")
... def process_pdf(file_path):
...     # Function implementation here
...     pass
...
>>> process_pdf("example.pdf")
[]
Raises:

Exception – Any exception raised by the wrapped function is caught, logged, and handled by returning an empty list.

nv_ingest_api.util.exception_handlers.schemas module#

nv_ingest_api.util.exception_handlers.schemas.schema_exception_handler(func, **kwargs)[source]#

A decorator that handles ValidationError exceptions for schema validation functions.

This decorator wraps a function that performs schema validation using Pydantic. If a ValidationError is raised, it logs detailed error messages and raises a ValueError with the combined error messages.

Parameters:

  • func (callable) – The function to be decorated. This function is expected to perform schema validation.

  • kwargs (dict) – Additional keyword arguments to be passed to the function.

Returns:

The wrapped function that executes func with exception handling.

Return type:

callable

Raises:

ValueError – If a ValidationError is caught, this decorator logs the error details and raises a ValueError with the combined error messages.

Notes

This decorator is particularly useful for functions that validate configurations or data models, ensuring that any validation errors are logged and communicated clearly.

Examples

>>> @schema_exception_handler
... def validate_config(config_data):
...     schema = MySchema(**config_data)
...     return schema
...
>>> try:
...     validate_config(invalid_config)
... except ValueError as e:
...     print(f"Caught error: {e}")
Caught error: Invalid configuration: field1: value is not a valid integer; field2: field required

Module contents#