nv_ingest_api.internal.primitives.nim package

Subpackages

• nv_ingest_api.internal.primitives.nim.model_interface package
  • Submodules
  • nv_ingest_api.internal.primitives.nim.model_interface.cached module
    • CachedModelInterface
      • format_input()
      • name()
      • parse_output()
      • prepare_data_for_inference()
      • process_inference_results()
  • nv_ingest_api.internal.primitives.nim.model_interface.decorators module
    • multiprocessing_cache()
  • nv_ingest_api.internal.primitives.nim.model_interface.deplot module
    • DeplotModelInterface
      • format_input()
      • name()
      • parse_output()
      • prepare_data_for_inference()
      • process_inference_results()
  • nv_ingest_api.internal.primitives.nim.model_interface.helpers module
    • get_model_name()
    • get_version()
    • is_ready()
    • preprocess_image_for_paddle()
  • nv_ingest_api.internal.primitives.nim.model_interface.nemoretriever_parse module
    • NemoRetrieverParseModelInterface
      • format_input()
      • name()
      • parse_output()
      • prepare_data_for_inference()
      • process_inference_results()
  • nv_ingest_api.internal.primitives.nim.model_interface.paddle module
    • PaddleOCRModelInterface
      • format_input()
      • name()
      • parse_output()
      • prepare_data_for_inference()
      • process_inference_results()
  • nv_ingest_api.internal.primitives.nim.model_interface.parakeet module
    • ParakeetClient
      • infer()
      • transcribe()
    • convert_to_mono_wav()
    • create_audio_inference_client()
    • process_transcription_response()
  • nv_ingest_api.internal.primitives.nim.model_interface.text_embedding module
    • EmbeddingModelInterface
      • format_input()
      • name()
      • parse_output()
      • prepare_data_for_inference()
      • process_inference_results()
  • nv_ingest_api.internal.primitives.nim.model_interface.vlm module
    • VLMModelInterface
      • format_input()
      • name()
      • parse_output()
      • prepare_data_for_inference()
      • process_inference_results()
  • nv_ingest_api.internal.primitives.nim.model_interface.yolox module
    • YoloxGraphicElementsModelInterface
      • name()
      • postprocess_annotations()
    • YoloxModelInterfaceBase
      • format_input()
      • parse_output()
      • postprocess_annotations()
      • prepare_data_for_inference()
      • process_inference_results()
      • transform_normalized_coordinates_to_original()
    • YoloxPageElementsModelInterface
      • name()
      • postprocess_annotations()
    • YoloxTableStructureModelInterface
      • name()
      • postprocess_annotations()
    • batched_overlaps()
    • bb_iou_array()
    • expand_boxes()
    • expand_chart_bboxes()
    • expand_table_bboxes()
    • find_boxes_inside()
    • find_matching_box_fast()
    • get_bbox_dict_yolox_graphic()
    • get_bbox_dict_yolox_table()
    • get_biggest_box()
    • get_weighted_box()
    • get_yolox_model_name()
    • match_with_title()
    • merge_boxes()
    • merge_labels()
    • postprocess_model_prediction()
    • postprocess_results()
    • prefilter_boxes()
    • resize_image()
    • weighted_boxes_fusion()
  • Module contents

Submodules

nv_ingest_api.internal.primitives.nim.default_values module

nv_ingest_api.internal.primitives.nim.nim_client module

class nv_ingest_api.internal.primitives.nim.nim_client.NimClient(

model_interface,

protocol: str,

endpoints: Tuple[str, str],

auth_token: str | None = None,

timeout: float = 120.0,

max_retries: int = 5,

)

Bases: object

A client for interfacing with a model inference server using gRPC or HTTP protocols.

close()

infer(data: dict, model_name: str, **kwargs) → Any

Perform inference using the specified model and input data.

Parameters:

• data (dict) – The input data for inference.

• model_name (str) – The model name.

• kwargs (dict) – Additional parameters for inference.

Returns:

The processed inference results, coalesced in the same order as the input images.

Return type:

Any

try_set_max_batch_size(model_name, model_version: str = '')

Attempt to set the max batch size for the model if it is not already set, ensuring thread safety.

nv_ingest_api.internal.primitives.nim.nim_model_interface module

class nv_ingest_api.internal.primitives.nim.nim_model_interface.ModelInterface

Bases: object

Base class for defining a model interface that supports preparing input data, formatting it for inference, parsing output, and processing inference results.

format_input(

data: dict,

protocol: str,

max_batch_size: int,

)

Format the input data for the specified protocol.

Parameters:

• data (dict) – The input data to format.

• protocol (str) – The protocol to format the data for.

name() → str

Get the name of the model interface.

Returns:

The name of the model interface.

Return type:

str

parse_output(

response,

protocol: str,

data: dict | None = None,

**kwargs,

)

Parse the output data from the model’s inference response.

Parameters:

• response (Any) – The response from the model inference.

• protocol (str) – The protocol used (“grpc” or “http”).

• data (dict, optional) – Additional input data passed to the function.

prepare_data_for_inference(data: dict)

Prepare input data for inference by processing or transforming it as required.

Parameters:

data (dict) – The input data to prepare.

process_inference_results(

output_array,

protocol: str,

**kwargs,

)

Process the inference results from the model.

Parameters:

• output_array (Any) – The raw output from the model.

• kwargs (dict) – Additional parameters for processing.

Module contents

class nv_ingest_api.internal.primitives.nim.ModelInterface

Bases: object

Base class for defining a model interface that supports preparing input data, formatting it for inference, parsing output, and processing inference results.

format_input(

data: dict,

protocol: str,

max_batch_size: int,

)

Format the input data for the specified protocol.

Parameters:

• data (dict) – The input data to format.

• protocol (str) – The protocol to format the data for.

name() → str

Get the name of the model interface.

Returns:

The name of the model interface.

Return type:

str

parse_output(

response,

protocol: str,

data: dict | None = None,

**kwargs,

)

Parse the output data from the model’s inference response.

Parameters:

• response (Any) – The response from the model inference.

• protocol (str) – The protocol used (“grpc” or “http”).

• data (dict, optional) – Additional input data passed to the function.

prepare_data_for_inference(data: dict)

Prepare input data for inference by processing or transforming it as required.

Parameters:

data (dict) – The input data to prepare.

process_inference_results(

output_array,

protocol: str,

**kwargs,

)

Process the inference results from the model.

Parameters:

• output_array (Any) – The raw output from the model.

• kwargs (dict) – Additional parameters for processing.

class nv_ingest_api.internal.primitives.nim.NimClient(

model_interface,

protocol: str,

endpoints: Tuple[str, str],

auth_token: str | None = None,

timeout: float = 120.0,

max_retries: int = 5,

)

Bases: object

A client for interfacing with a model inference server using gRPC or HTTP protocols.

close()

infer(data: dict, model_name: str, **kwargs) → Any

Perform inference using the specified model and input data.

Parameters:

• data (dict) – The input data for inference.

• model_name (str) – The model name.

• kwargs (dict) – Additional parameters for inference.

Returns:

The processed inference results, coalesced in the same order as the input images.

Return type:

Any

try_set_max_batch_size(model_name, model_version: str = '')

Attempt to set the max batch size for the model if it is not already set, ensuring thread safety.